[antlr-interest] C and #define

[Gavin Lambert]

At 10:43 22/05/2007, Paul Keir wrote:
>That certainly does give me something to think about. For one 
>thing, I haven't seen the $type construct before; I guess it 
>redirects the rule clause to another lexer rule called 'DEFINE'.

Not quite; it simply reassigns the type of the token being 
generated by the current rule.  This is how you can have common 
grammar creating multiple tokens.  Instead of having a separate 
rule, it's simply declared in the tokens header section and 
assigned as needed through actions.  Have a look at some of the 
other example grammars; I'm sure several of them use this 
technique.

>It seems like you and Gavin are both suggesting methods which 
>handle whitespace explicitly. i.e. Not assigned to the HIDDEN 
>channel. I'm still learning antlr, but I guess this would require 
>rewriting every rule of my grammar. I don't mind a day or so's 
>work, but I'll have to think through the implications a little 
>more.

No, that's not the case.  These are all being done in *lexer* 
rules, and inside the lexer all the whitespace is always present 
and must always be accounted for if it's contained within a single 
token (since the lexer works character-by-character).  Whitespace 
outside the other tokens, on the other hand, gets consumed by the 
top-level WS token (or whatever you've called it), which is 
usually then assigned to the hidden channel.  This means that the 
*parser* rules don't have to worry about the whitespace, since 
they'll never see it.

I think I phrased that poorly, but I hope you get what I mean :)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20070522/0366e648/attachment.html