[antlr-interest] best approach for macro substitution

[Stanislav Sokorac]

Two approaches that I think are the most effective are pre-processing and
substitution in the lexer. You could also do it in the parser (see "handling
include files" FAQ), although it makes your AST/walker more complicated.
The advantage of pre-processing is the flexibility of having macros anywhere
in the code, even as parts of tokens. A disadvantage is that you have to
keep track of all macro replacements and then do some post-processing on all
line and character numbers to be able to report correct numbers for the
original file. Also, you have to pass through the file twice.

Doing the replacement in the lexer is probably the most effective solution
for the majority of the cases. As long as your macros are not allowed to
define parts of tokens (or you can modify your lexer to make it so), then
this is a pretty easy solution. Whenever you detect a macro, drop that
token, switch the input stream to the macro value, and then switch back when
it's done. You'll need to detect macro definitions and keep a map of them,
of course.

Stan

On Wed, Sep 2, 2009 at 10:05 AM, Gabriel Erzse <gabriel_erzse at yahoo.com>wrote:

> Hi,
>
> What do you think would be the best approach to solve macro substitution
> with ANTLR?
>
> One option I've seen is to build it in the lexer. I've seen a sample
> grammar for a C preprocessor on the website, but I didn't go in detail into
> it. My language is not C, but a custom language that resembles more to Java.
>
> Another option I'm thinking of is to do the macro replacement after the AST
> is built? Basically walk over the tree, detect macro patterns in the
> subtrees and do needed replacements. But this one sounds messy, with all the
> involved node manipulations.
>
> Another one would be to integrate some external framework. For example
> there is the GNU M4 framework. Don't know if there is something similar that
> can be easily plugged into ANTLR.
>
> If someone has experience with macro handling, I could use any suggestion.
>
> Thanks,
> Gabi.
>
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20090902/b1246490/attachment.html