[antlr-interest] C and #define
Gavin Lambert
antlr at mirality.co.nz
Tue May 22 01:19:38 PDT 2007
At 10:43 22/05/2007, Paul Keir wrote:
>That certainly does give me something to think about. For one
>thing, I haven't seen the $type construct before; I guess it
>redirects the rule clause to another lexer rule called 'DEFINE'.
Not quite; it simply reassigns the type of the token being
generated by the current rule. This is how you can have common
grammar creating multiple tokens. Instead of having a separate
rule, it's simply declared in the tokens header section and
assigned as needed through actions. Have a look at some of the
other example grammars; I'm sure several of them use this
technique.
>It seems like you and Gavin are both suggesting methods which
>handle whitespace explicitly. i.e. Not assigned to the HIDDEN
>channel. I'm still learning antlr, but I guess this would require
>rewriting every rule of my grammar. I don't mind a day or so's
>work, but I'll have to think through the implications a little
>more.
No, that's not the case. These are all being done in *lexer*
rules, and inside the lexer all the whitespace is always present
and must always be accounted for if it's contained within a single
token (since the lexer works character-by-character). Whitespace
outside the other tokens, on the other hand, gets consumed by the
top-level WS token (or whatever you've called it), which is
usually then assigned to the hidden channel. This means that the
*parser* rules don't have to worry about the whitespace, since
they'll never see it.
I think I phrased that poorly, but I hope you get what I mean :)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20070522/0366e648/attachment.html
More information about the antlr-interest
mailing list