[antlr-interest] How to do preprocessing in antlr v4?

[Douglas Godfrey]

I implemented a C pre-processor for a C like language entirely in the
Lexer. You can too if you want to.

1)	#define	was limited to use in the preprocessor and to setting the text
of identifiers.
2)	#define values were stored in a map<string,string>
3)	#define macros were implemented by a macro re-write rule that generated
and included a memory char stream
4)	text concatenation and length attributes ware not supported but they
could have been.
5)		#include was implemented by a nested char stream.
6)	#if, #elif, #else, #ifdef, #ifndef all set the top of a boolean stack
to the condition value,
	if any of the values on the stack are false then the channel for
subsequent tokens was set to $hidden.

All preprocessor actions were handled directly in the lexer without
needing to emit a text stream.
The parser just sees the the resulting [non-hidden] tokens.



On 11/18/12 9:52 PM, "Martin d'Anjou" <martin.danjou14 at gmail.com> wrote:

>On 12-11-18 09:51 AM, Bernard Kaiflin wrote:
>>
>> Having said that, if your need is doing some preprocessing before
>> parsing the C program, you would need a full C grammar combined with a
>> full C preprocessor, and maybe use the TokenStreamRewriter feature
>> described in paragraphs Rewriting the Input Stream on page 54 and
>> Accessing Hidden Channels on page 208 of the beta 3 book.
>>
>
>I have reached page 54 of the book (awesome book by the way), and had a
>peek at page 208. If I want to implement a preprocessor using the
>TokenStreamRewriter, I need the parser to call the listener when it
>reaches a rule in the parser, which is impossible because there are no
>preprocessor rules in the parser. So now I have to think about the other
>parts of the answers that you offered.
>
>Thanks,
>Martin
>
>
>List: http://www.antlr.org/mailman/listinfo/antlr-interest
>Unsubscribe: 
>http://www.antlr.org/mailman/options/antlr-interest/your-email-address