[antlr-interest] Syntactic predicate in lexer rule

[Terence Parr]

On Jan 2, 2006, at 10:34 PM, Martin Traverso wrote:
> Unfortunately, it doesn't. The lexer is using a cyclic DFA to  
> predict what to match, and doesn't seem to take advantage of the  
> predicate. In fact, the predicate is not even referenced anywhere  
> in the lexer (the code for it is generated, though).
>
> So, I tried with the following grammar, which does have two alts to  
> choose from:
>
> grammar T;
>
> tokens {
>     INTEGER;
>     FLOAT;
> }
>
> range:  number ('..' number)?;
> number: FLOAT | INTEGER;
>
> NUMBER:  (DIGITS '.' DIGITS) => DIGITS '.' DIGITS { type = FLOAT; }
>                 | DIGITS { type = INTEGER; };
>
> fragment
> DIGITS: ('0'..'9')+;
>
>
> But the cyclic DFA doesn't look right:
>
> s0 -> s1     upon '0'..'9'
> s1 -> s1     upon '0'..'9'
> s1 -> (s4)   upon '.'  => alt = 1
> s1 -> (s2)   otherwise => alt = 2

Actually, that is correct.  There is no ambiguity.  A FLOAT *must*  
have a '.' and so LL(*) simply looks for that; else it's an INT.  Sem  
preds are not used unless they are needed to resolve a syntactic issue.

Ter