[antlr-interest] best approach for macro substitution

Daniels, Troy (US SSA) troy.daniels at baesystems.com
Wed Sep 2 07:37:57 PDT 2009


If it is a c-style preprocessor, once you've built the AST will be too
late.  Consider this example:

// I like pascal and capitals
#define BEGIN {
#define END }
#define IF if


IF ( x > 0 )
BEGIN
  x = -1;
END

A typical java parser isn't going to be able to recognize most of that,
even though it's a perfectly valid code fragment once the macros are
replaced.

Building it into the lexer is also challenging, assuming full
replacements are allowed:

#define IF_POSITIVE(x, body) if ((x) > 0) { body; }

You would need a lexer rule that, after this statement is encountered,
would recognize this expression and then generate an arbitrarily complex
stream of tokens. 

It's not impossible, but it is rather complicated.  If you have a
separate preprocessor, it may be simpler to try parsing the processed
text. 

Troy

> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of Gabriel Erzse
> Sent: Wednesday, September 02, 2009 10:05 AM
> To: antlr-interest at antlr.org
> Subject: [antlr-interest] best approach for macro substitution
> 
> Hi,
> 
> What do you think would be the best approach to solve macro
substitution
> with ANTLR?
> 
> One option I've seen is to build it in the lexer. I've seen a sample
> grammar for a C preprocessor on the website, but I didn't go in detail
> into it. My language is not C, but a custom language that resembles
more
> to Java.
> 
> Another option I'm thinking of is to do the macro replacement after
the
> AST is built? Basically walk over the tree, detect macro patterns in
the
> subtrees and do needed replacements. But this one sounds messy, with
all
> the involved node manipulations.
> 
> Another one would be to integrate some external framework. For example
> there is the GNU M4 framework. Don't know if there is something
similar
> that can be easily plugged into ANTLR.
> 
> If someone has experience with macro handling, I could use any
suggestion.
> 
> Thanks,
> Gabi.
> 
> 
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address



More information about the antlr-interest mailing list