[antlr-interest] Basic predicate question

Terence Parr parrt at cs.usfca.edu
Tue May 20 10:26:46 PDT 2008


On May 19, 2008, at 5:51 AM, Haralambi Haralambiev wrote:

> I checked the grammar above with the latest antlrworks and it says  
> that the NOT token is not reachable. So, at least now it gives you a  
> warning, but I still think that this is a bug and that the  
> predicates should be taken into account!

The issue with this grammar is that NOT is not protected by a gated  
predicate. Try:

NOT      :      {getCharPositionInLine()>0}?' => NOT';

>
Ter
>
> Hari
>
> On 5/16/08, Haralambi Haralambiev <hharalambiev at gmail.com> wrote:  
> Hello,
>
> I have stumbled upon on a problem where the DFA's prediction is  
> incorrect, as it does not take into account the predicate for a rule.
>
> Consider the following grammar:
> -----------------------------------------------------
> lexer grammar test;
>
> IF	 :	{getCharPositionInLine()==0}? => 'IF';	
> UNKNOWN_CMD	:	{getCharPositionInLine()==0}? => Identifier;
>
> NOT	 :	'NOT';	
>
> Identifier	:	('A'..'Z'|'a'..'z')+;
> WS	:	('\t'|' ')+;
> -----------------------------------------------------
>
> In the language I am working on, the first word is always some  
> command. As I am not interested in all the commands, I decided to  
> define an UNKNOWN_CMD.
>
> I also have some other keywords that are not commands, thus not in  
> the beginning of the line, e.g. the described above NOT.
>
> With the grammar above and the input "IF NOT SOMETHING", the lexer  
> fails, as the DFA has predicted that the "NOT" part of the stream  
> should be an UNKNOWN_COMMAND.
> My understanding was that predicates are somehow included when  
> predicting the next token and thus I am confused why the lexer will  
> attempt to match the "NOT" to this lexer rule?
>
> If we put the NOT rule above the unknown rule, everything will work  
> properly, but this is a work around and I would like to understand  
> the cause of the issue.
>
> Could someone point the reason of such a behaviour?
>
> Thanks,
> Hari
>



More information about the antlr-interest mailing list