[antlr-interest] Syntactic predicate in lexer rule

Martin Traverso mtraverso at gmail.com
Mon Jan 2 22:34:05 PST 2006


> Actually, I take this back.  There is a quirk i just realized.  Syn
> preds are converted to sem preds (at least for now) for
> implementation, hence, they will be hoisted like any other
> predicate.  The (DIGITS '.' DIGITS) is converted to a boolean test
> and hoisted into the artificial Tokens rule and that might work.


Unfortunately, it doesn't. The lexer is using a cyclic DFA to predict what
to match, and doesn't seem to take advantage of the predicate. In fact, the
predicate is not even referenced anywhere in the lexer (the code for it is
generated, though).

So, I tried with the following grammar, which does have two alts to choose
from:

grammar T;

tokens {
    INTEGER;
    FLOAT;
}

range:  number ('..' number)?;
number: FLOAT | INTEGER;

NUMBER:  (DIGITS '.' DIGITS) => DIGITS '.' DIGITS { type = FLOAT; }
                | DIGITS { type = INTEGER; };

fragment
DIGITS: ('0'..'9')+;


But the cyclic DFA doesn't look right:

s0 -> s1     upon '0'..'9'
s1 -> s1     upon '0'..'9'
s1 -> (s4)   upon '.'  => alt = 1
s1 -> (s2)   otherwise => alt = 2


Martin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20060102/e8bfd9f9/attachment.html


More information about the antlr-interest mailing list