[antlr-interest] Tokens
David-Sarah Hopwood
david-sarah at jacaranda.org
Fri Nov 27 17:28:47 PST 2009
Ronald Sok wrote:
> Kevin J. Cummings wrote:
>> You are close. What you have here is keywords as opposed to reserved
>> words. When implementing the former, you will need to do something like
>> (at least this is what I do using ANTLR 2.7.7):
>>
>> id : ID
>> | k:keyword
>> { #k->setType(ID); }
>> // This changes the token type of a keyword to an ID
>> ;
>>
>> keyword
>> : APPLE | PEAR | ORANGE
>> ;
>>
>> someName
>> : 'Name:' id NEWLINE
>> ;
[...]
> Ok, I tried to change this into ANTLR 3 syntax, but ran into the fact that
> the result of keyword is a subtype of ParserRuleReturnScope, which
> definitely is not a Token type. So trying to change the type of the token
> like how you showed in ANTLR 2.7.7 syntax does not work for me.
> This is what I have, which results in an error while generating the grammar.
>
> id : ID
> | keyword
> { $type=ID; }
> // from http://www.antlr.org/wiki/display/ANTLR3/Migrating+from+ANTLR+2+to+ANTLR+3
> ;
It's not necessary to change the type of the token unless you're actually
going to depend on that type in some other rule that uses 'id'. I.e. just
omitting '{ $type = ID; }' will work in most situtations.
If you do want to change it, then that can't be done in this case using
'$type =', which only works in a lexer rule. To change the type of the
token that is at the root of a particular subtree in a parser rule, use:
id : ID
| k=keyword { if ($k.tree != null) $k.tree.getToken().setType(ID); }
;
(Yes, it's a bit ugly. It is possible for $k.tree to be null in certain
error cases, unfortunately. If anyone has a way of doing this that is less
ugly but always works, I'd like to hear it.)
--
David-Sarah Hopwood ⚥ http://davidsarah.livejournal.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 292 bytes
Desc: OpenPGP digital signature
Url : http://www.antlr.org/pipermail/antlr-interest/attachments/20091128/51f05225/attachment.bin
More information about the antlr-interest
mailing list