[antlr-interest] handling preprocessor #define and macro substitutions

thrutchy eric_mahurin at yahoo.com
Sun Jul 18 13:45:15 PDT 2004


I'm parsing a language that has something similar to the C
preprocessor constructs.  Is there something in antlr already out
there for this that somebody can point me to?

My main issue is how to deal with #define statements and the
substitutions associated.  Here are the issues/questions I can think of:

- the substitution can occur within two different lexer states where
the tokens are different, so I think I need to store the
substitutition text as a string as opposed to a list of tokens
- how to make the lexer input be a string instead of a file.  Oh, I
just figured that out while typing this - isn't there a stringstream
or something like that in java?
- during the macro substitution, handling parameter substituion.  One
of my lexer states that doesn't even have an identifier defined could
make this problematic.
- expressions as arguments to the macro call.  The expression is
defined in the parser and not the lexer, so its definition is not
there.  Right now, I'm just matching up grouping symbols that can
contain the "," which is also the separator between the expressions.

Secondarily, an example for handling #if/#else would be nice.  I
assume this would just gobble up tokens when it would find the
condition to be false (nextToken).

Or, should the preprocessor be a separate filter instead of included
with the lexer - token stream filter?

Again a preprocessor example with text subtitution (including
parameters) and conditional constructs would help out a lot.  There's
got to one out there.  I did some googling without success.

Eric




 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
    antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 



More information about the antlr-interest mailing list