[antlr-interest] Understanding priorities in lexing (newbie)

[Terence Parr]

On Jul 12, 2007, at 11:14 AM, Thomas Brandon wrote:
> If you change the grammar to:
> grammar lex2;
> start
> 	:	PROGRAM;
>
> PROGRAM : (  KEYWORD  |  OTHER  )+;
>
> fragment KEYWORD : 'a' 'b' 'c';
>
> fragment OTHER : 'a' | 'b' | 'c';
>
> Then in PROGRAM you get a predictor that checks for 'abc' before
> deciding that it is a KEYWORD.
> Is it not possible to have the next token predictor function like  
> this?

Hi Tom.  Actually even if I did, OTHER OTHER matches 'ab' as does  
KEYWORD and so it has to resolve the ambiguity, which it does in  
favor of first rule specified.  It uses PROGRAM rule w/o the +  
because what if you had an error char?  There is an implied loop to  
PROGRAM in nextToken() method.

That said, i understand that your thinking is more in line with the  
most natural interpretation.

Ter