[antlr-interest] What's the best way to differentiate identifiers and keywords in the lexer ?

Anthony Brenelière abreneliere at telys.com
Mon Jan 27 04:25:59 PST 2003


What's the best way to differentiate identifier and keywords in the
lexer ?

I have read the following solutions to avoid nondeterminism :

--------
1. using the token list, or use the strings "(keyword)" in the parser,
for keywords.

..but the problem is that i need some rule to assign code to execute,
for each keyword.
--------
2. using the syntactic predicates in the lexer

..but the problem is that i have to send back a TOKEN that is not the
TOKEN od the keyword itself.

I would have something like:

KEY_OR_ID : (KEYWORD1)=> KEYWORD1 | ... | (KEYWORDn)=> KEYWORDn
;

ID : ('a'..'z'|'A'..'Z'|'_')
;

protected KEYWORD1 : "KEYWORD1" { my code 1 } ;
(...)
protected KEYWORDn : "KEYWORDn" { my code n } ;

..but I could not return the KEYWORDi token to the parser.
---------


Is there another third (good) solution with ANTLR ?

Cordially,
Anthony B.


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 



More information about the antlr-interest mailing list