[antlr-interest] Tokens

[David-Sarah Hopwood]

Steve Bennett wrote:
>> APPLE
>>    :    'Apple'
>>    ;
>>
>> PEAR
>>    :    'Pear'
>>    ;
>>
>> ORANGE
>>    :    'Orange'
>>    ;
>>
>> ID  :    ('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'0'..'9'|'_')*
>>    ;
> 
> Kevin's answer is probably the right one, but what I have done sometimes is:
> 
> ID: ('a'..'z'|'A'..'Z'|'_'|APPLE|PEAR|ORANGE)
> ('a'..'z'|'A'..'Z'|'0'..'9'|'_'|APPLE|PEAR|ORANGE)*
> 
> If I'm not misremembering how lexing works...it's been a little while.

That won't work. Lexer rules are matched in order of their appearance
in the grammar, first being highest priority, so 'Apple' will match
APPLE and not ID regardless of the change above. Actually the second
version of ID is completely equivalent to the first.

-- 
David-Sarah Hopwood  ⚥  http://davidsarah.livejournal.com

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 292 bytes
Desc: OpenPGP digital signature
Url : http://www.antlr.org/pipermail/antlr-interest/attachments/20091127/e439988f/attachment.bin