[antlr-interest] How to do preprocessing in antlr v4?
Douglas Godfrey
douglasgodfrey at gmail.com
Mon Nov 19 05:26:23 PST 2012
I implemented a C pre-processor for a C like language entirely in the
Lexer. You can too if you want to.
1) #define was limited to use in the preprocessor and to setting the text
of identifiers.
2) #define values were stored in a map<string,string>
3) #define macros were implemented by a macro re-write rule that generated
and included a memory char stream
4) text concatenation and length attributes ware not supported but they
could have been.
5) #include was implemented by a nested char stream.
6) #if, #elif, #else, #ifdef, #ifndef all set the top of a boolean stack
to the condition value,
if any of the values on the stack are false then the channel for
subsequent tokens was set to $hidden.
All preprocessor actions were handled directly in the lexer without
needing to emit a text stream.
The parser just sees the the resulting [non-hidden] tokens.
On 11/18/12 9:52 PM, "Martin d'Anjou" <martin.danjou14 at gmail.com> wrote:
>On 12-11-18 09:51 AM, Bernard Kaiflin wrote:
>>
>> Having said that, if your need is doing some preprocessing before
>> parsing the C program, you would need a full C grammar combined with a
>> full C preprocessor, and maybe use the TokenStreamRewriter feature
>> described in paragraphs Rewriting the Input Stream on page 54 and
>> Accessing Hidden Channels on page 208 of the beta 3 book.
>>
>
>I have reached page 54 of the book (awesome book by the way), and had a
>peek at page 208. If I want to implement a preprocessor using the
>TokenStreamRewriter, I need the parser to call the listener when it
>reaches a rule in the parser, which is impossible because there are no
>preprocessor rules in the parser. So now I have to think about the other
>parts of the answers that you offered.
>
>Thanks,
>Martin
>
>
>List: http://www.antlr.org/mailman/listinfo/antlr-interest
>Unsubscribe:
>http://www.antlr.org/mailman/options/antlr-interest/your-email-address
More information about the antlr-interest
mailing list