[antlr-interest] how to deal with the number and identifier tokens overlap problem?

Fan Yang fanyang_32 at yahoo.ca
Thu May 5 08:45:05 PDT 2005


Hi,
 
I encountered the number and identifier tokens overlap problem, for example, 1000-MAIN, the lexer will return INTEGER 1000 and a parsing error instead of "1000-MAIN" as IDENTIFIER. 
 
INTEGER:
('0'...'9')+
;
 
IDENTIFIER:
(('a'..'z')|('0'..'9'))+ (('-')+ (('a'..'z')|('0'..'9'))+)*
;
 
After researching for a while on anltr news group, I know that I can use left factoring or increase lookahead characters(not for this case) to fix this problem. I tried to use syntax predicate as showing in the following code snippet, but the lexer seems running for ever, never return.
 
IDENTIFIER:
( (('a'..'z')|('0'..'9'))+ (('-')+ (('a'..'z')|('0'..'9'))+)*)=>

 (('a'..'z')|('0'..'9'))+ (('-')+ (('a'..'z')|('0'..'9'))+)*
|
 ('0'...'9')+ {_ttype = INTEGER;}

;
 
Would you please tell me how to fix this kind problem? Any suggestion will be appreciated.
 
Fan
 
 



---------------------------------
Post your free ad now! Yahoo! Canada Personals
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20050505/80af3b53/attachment.html


More information about the antlr-interest mailing list