[antlr-interest] Basic predicate question

[Haralambi Haralambiev]

Hello,

I have stumbled upon on a problem where the DFA's prediction is incorrect,
as it does not take into account the predicate for a rule.

Consider the following grammar:
-----------------------------------------------------
lexer grammar test;

IF : {getCharPositionInLine()==0}? => 'IF';
UNKNOWN_CMD : {getCharPositionInLine()==0}? => Identifier;

NOT : 'NOT';

Identifier : ('A'..'Z'|'a'..'z')+;
WS : ('\t'|' ')+;
-----------------------------------------------------

In the language I am working on, the first word is always some command. As I
am not interested in all the commands, I decided to define an UNKNOWN_CMD.

I also have some other keywords that are not commands, thus not in the
beginning of the line, e.g. the described above NOT.

With the grammar above and the input "IF NOT SOMETHING", the lexer fails, as
the DFA has predicted that the "NOT" part of the stream should be an
UNKNOWN_COMMAND.
My understanding was that predicates are somehow included when predicting
the next token and thus I am confused why the lexer will attempt to match
the "NOT" to this lexer rule?

If we put the NOT rule above the unknown rule, everything will work
properly, but this is a work around and I would like to understand the cause
of the issue.

Could someone point the reason of such a behaviour?

Thanks,
Hari
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20080516/ea02d7f4/attachment.html