[antlr-interest] token not recognized

[John B. Brodie]

Greetings!

On Thu, 2010-04-22 at 12:04 -0800, kumarr wrote:
> Hi all,
> 
> I have defined a token called GEN that can take on various string values.
> The grammar is below.
> 
> The token is question is GEN. I'm trying to parse a very simple string
> conforming to this grammar: "Unavailable(LEP)". When it sees "LEP" it
> complains that it found 'LEP' when it was expecting GEN. The error message
> is:
> 
> line 1:13 mismatched input 'LEP' expecting GEN
> 
> I don't understand this. I have defined "LEP" as one of the values that GEN
> can take!!
> 
> The same problem happens for "Unavailable(REP)". The inputs
> "Unavailable(VFSG_L1)", "Unavailable(VFSG_L2)" etc are parsed without
> errors.
> 
> does anyone know what could be gong on?
> Thanks.
> Rajesh
> 
> **********************grammar		 Expression2; 
> 
> prog	
> 	:	exp
> 
> 	;
> 	
> exp	
> 	: atomic_expr 
> 	| 
> 	;
> 	
> 
> atomic_expr 
> 	:	 ID 
> 	|	ID '>' NUM 
> 	|	ID '<' NUM 
> 	|	'Unavailable' '(' GEN ')'
> 	;
> 
> ID	:	 ('a'..'z'|'A'..'Z')('a'..'z'|'A'..'Z'|'0'..'9')*
> 	;
> 
> NUM	:	('0'..'9')+
> 	;
> 	
> GEN	:     'VFSG_L1'
> 	| 	'LEP'
> 	| 	'REP'
> 	|	'VFSG_L2'
> 	|	'VFSG_R1'
> 	|	'VFSG_R2'
> 	|	'ASG_L'
> 	|	'ASG_R' 
> 	;
> 	
> BUS	:	'L1_V235_bus'
> 	|	'L2_V235_bus'
> 	|	'R1_V235_bus'
> 	|	'R2_V235_bus'
> 	|	'Backup_bus'
> 	;
> 	
> WS	:	(' '|'\t'|'\n')+ {skip();};
> 
> *****************************************

Your ID rule overlaps with the LEP and REP keywords (but not with any
keyword that has an _ in it).

Move your ID rule to be after the keyword rules.

When two lexer rules match the same input string, ANTLR lexers select
the Token that appears first in the set of rules....

'Unavailable' worked because ANTLR puts parser created tokens implicitly
first in the set of lexer rules.

Hope this helps.
   -jbb