[antlr-interest] Tokens
David-Sarah Hopwood
david-sarah at jacaranda.org
Fri Nov 27 17:39:23 PST 2009
Kevin J. Cummings wrote:
> On 11/27/2009 06:05 PM, Ronald Sok wrote:
>> Kevin J. Cummings wrote:
>>> You are close. What you have here is keywords as opposed to reserved
>>> words. When implementing the former, you will need to do something like
>>> (at least this is what I do using ANTLR 2.7.7):
>>>
>>> id : ID
>>> | k:keyword
>>> { #k->setType(ID); }
>>> // This changes the token type of a keyword to an ID
>>> ;
>>>
>>> keyword
>>> : APPLE | PEAR | ORANGE
>>> ;
>>>
>>> someName
>>> : 'Name:' id NEWLINE
>>> ;
>>>
>>> You could reduce the number of productions by folding, but the principal
>>> of changing the token type of keywords is what is important here. And
>>> you may have to find out how to do this with ANTLR 3.x.
>>
>> Ok, I tried to change this into ANTLR 3 syntax, but ran into the fact that
>> the result of keyword is a subtype of ParserRuleReturnScope, which
>
> Sorry, my bad, should be KEYWORD and done in the lexer, not the parser!
That won't work because either:
- KEYWORD is before APPLE, PEAR and ORANGE, in which case it always takes
priority and the type of a Token will never be APPLE, PEAR or ORANGE;
- or, KEYWORD is after APPLE, PEAR and ORANGE, in which case those rules
take priority and the type of a Token will never initially be KEYWORD.
You could override it, but if you do that in a lexer rule then you
don't have enough context to determine what it should be (and '$type ='
can't be used in a parser rule).
As I said in my other followup, it's usually not necessary to change the
type (but you can do so using the code given in that post if you want).
--
David-Sarah Hopwood ⚥ http://davidsarah.livejournal.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 292 bytes
Desc: OpenPGP digital signature
Url : http://www.antlr.org/pipermail/antlr-interest/attachments/20091128/b838671c/attachment.bin
More information about the antlr-interest
mailing list