[antlr-interest] Basic predicate question

[Haralambi Haralambiev]

I checked the grammar above with the latest antlrworks and it says that the
NOT token is not reachable. So, at least now it gives you a warning, but I
still think that this is a bug and that the predicates should be taken into
account!

Hari

On 5/16/08, Haralambi Haralambiev <hharalambiev at gmail.com> wrote:
>
> Hello,
>
> I have stumbled upon on a problem where the DFA's prediction is incorrect,
> as it does not take into account the predicate for a rule.
>
> Consider the following grammar:
> -----------------------------------------------------
> lexer grammar test;
>
> IF : {getCharPositionInLine()==0}? => 'IF';
> UNKNOWN_CMD : {getCharPositionInLine()==0}? => Identifier;
>
> NOT : 'NOT';
>
> Identifier : ('A'..'Z'|'a'..'z')+;
> WS : ('\t'|' ')+;
> -----------------------------------------------------
>
> In the language I am working on, the first word is always some command. As
> I am not interested in all the commands, I decided to define an UNKNOWN_CMD.
>
> I also have some other keywords that are not commands, thus not in the
> beginning of the line, e.g. the described above NOT.
>
> With the grammar above and the input "IF NOT SOMETHING", the lexer fails,
> as the DFA has predicted that the "NOT" part of the stream should be an
> UNKNOWN_COMMAND.
> My understanding was that predicates are somehow included when predicting
> the next token and thus I am confused why the lexer will attempt to match
> the "NOT" to this lexer rule?
>
> If we put the NOT rule above the unknown rule, everything will work
> properly, but this is a work around and I would like to understand the cause
> of the issue.
>
> Could someone point the reason of such a behaviour?
>
> Thanks,
> Hari
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20080519/0356fc63/attachment.html