[antlr-interest] How to do preprocessing in antlr v4?

[Mike Lischke]

Hey Martin,

> What is the right approach to implement preprocessor directives 
> (#define, #undef, #ifdef,, #ifndef, #if, #elsif, #else, #endif, 
> #include, and user defined macros)  in Antlr v4?
> 
> I would find it a waste to write a separate preprocessor to tokenize the 
> input stream and resolve the directives and macros (and their 
> parameters), just to write the token stream back to a file, then 
> retokenize with antlr... It looks to me that preprocessor does almost 
> the same work as the lexer.
> 
> Macro parameters are the reason why I want to tokenize the input to the 
> preprocessor. So I want lexer -> preprocessor -> parser.


In 2004 I created a project to convert Windows resource files (.rc, which share many aspects with C/C++ header files) to XML. I used ANTLR (2.7 at that time) to generate lexer and parsers (the project is completely written in Java). My approach was to use a special input stream that does the preprocessing, so there is no need to write the preprocessed input to a separate file. So the chain is input->preprocessor->lexer->parser.

More details here: http://www.antlr.org/pipermail/antlr-interest/2005-December/014767.html

Note: WebSVN is no longer available, however you can download a zip from the download section for all my projects: http://www.soft-gems.net/index.php/all-downloads (I have yet to create a dedicated document describing the converter).

HTH, 

Mike
-- 
www.soft-gems.net