[antlr-interest] Implementing Preprocessor Functions

[John Green]

1. Don't.
2. If it's not your decision anyway, then at least make sure there is an impenetrable barrier between your preprocessor and your lexer. In other words, the preprocessor supplies a character stream to the lexer, and there is no other communication between the two.

I wrote a parser for a pre-existing language (see joanju.com) and the preprocessor was the biggest hurdle. Not only did the preprocessing make it difficult for me to build the parser, but it continues to wreck havoc on all of the benefits that one would expect to gain from a parser, such as documentation generation, pretty printing, lint, and refactoring. No doubt it also makes it messy to build debuggers.

Macro-level processing is a very poor workaround for shortcomings in a language and its compiler.

John


de Jong, Ron wrote:
> Hi,
> 
> In the scripting language we’ve implemented we need to add similar 
> functionality to the C++ #include, #ifdef, #else, #endif, #define.
> 
> I have already implemented #include by using a tokenstream filter – this 
> seems to work well.
> 
> Question: is this the best way of implementing preprocessing? Also does 
> anyone have suggestions about how to implement the other preprocessing 
> functions?
> 
> Cheers,
> Ron