[antlr-interest] Q: how to incorporate a preprocessor in the flow?

[A Z]

Hi Martin,

  I just completed an SV preprocessor which can parse UVM 1.0 successfully.
After 2 revisions I settled on a completely separate preprocessor(lexer and
parser). As you saw, you need to tokenize the macro_text in order to easily
support macros with arguments and detect the three escaped tokens `", `\`"
and ``. I'm not sure how well a lexer only approach could handle cases where
a macro substitution can merge text with a previously lexed token. The
separate approach still has flaws, such as good error reporting. Of course I
could be missing an obvious easy solution.



On Sun, Apr 3, 2011 at 9:51 PM, Martin d'Anjou <point14 at magma.ca> wrote:

> Hello,
>
> I am trying to find a way to incorporate a preprocessor in the ANTLR
> flow. I thought of doing this before the lexer, but I need to tokenize
> the incoming char stream for macro substitution to be easy. I thought of
> doing it between the lexer and the parser, and replace the preprocessor
> tokens with their expansion before feeding the token stream to the
> parser, so I guess I would end up using something like the
> TokenRewriteStream??? Can someone steer me in the right direction
> please? Or should I be using lexer rule actions? In which case, any
> example on how to access the token stream of the replacement token list
> of an identifier? Too many questions sorry.
>
> The language I am hoping to tokenize is SystemVerilog and has C-like
> preprocessor macros (`include, `ifdef, `define NAME(params,...), token
> concatenation, etc.).
>
> Regards,
> Martin
>
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>