[antlr-interest] behaviour of lexer

[Philippe Frankson]

Hi all,
I'm using ANTLR 3.1.2.
Let's make it simple:
@int('444') is a function converting a string into integer.
If I don't have parentheses, then it is not a function, it is only a column name. Ex.: @test, @integer, @in, ....

Here is a part of lexer rules:

fragment DIGIT	: ('0'..'9');
fragment ALPHA	: ('a'..'z'|'A'..'Z'|'_');

OB		: '(';
INTTOKEN	: ('@int' OB)=> '@int'; // so I check if there is an open parenthesis to return INTTOKEN.
AT		: '@';
NAME		: ALPHA (ALPHA | DIGIT)*;


If my input is:
--> @int('444'), 	INTTOKEN is returned, which is OK.
--> @integer,	mismatched input 'eger', expecting '('. 	--> I would expect AT token followed by NAME token.
--> @int,		mismatched input '', expecting '('.		--> same remark.
--> @i,		mismatched character '', expecting 'n'. 	--> same remark.


Now, instead of AT and NAME rule, I replace by:
ATNAME	: '@' NAME;

If my input is:
--> @int('444'), 	INTTOKEN is returned, which is OK.
--> @integer,	ATNAME is returned, which is OK.
--> @int,		mismatched input '', expecting '('. --> This is what I want but I don't understand why I don't get ATNAME.
--> @i,		ATNAME is returned, which is OK.

So, I have 2 questions:
1) Why isn't it working when separating the 2 lexer rules (AT and NAME) ?
2) Why isn't it returning ATNAME token when input is '@int' in the second case ?

Thanks a lot for any help
Philippe Frankson





List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address