[antlr-interest] Antlr 3 Lexer problem
Gavin Lambert
antlr at mirality.co.nz
Wed Jun 27 00:43:51 PDT 2007
At 09:06 27/06/2007, Geoffrey Zhu wrote:
>The syntactic predicate does not seem to work. The lexer chokes
on
>exactly the same location 'c' if I pass in "( security".
>
>In mTokens() it still looks ahead only one step to determine
what
>should e the next token.
I think this is another occurrence of the case that Ter claims is
by design, but myself and a few others would like to be different:
the lexer doesn't do backtracking, it simply fails with
NoViableAltExceptions (or the equivalent) -- even when the parent
grammar does do backtracking. Basically once it enters a
particular token it's going to either match that token or cause an
error; it won't go back and pick a different token.
It is usually possible to work around this by jumping through
hoops backwards with your eyes closed (aka reordering rules and/or
juggling predicates, parentheses, and fragment rules until
something clicks, or just giving up and moving some of it over to
the parser), but it's definitely not intuitive.
More information about the antlr-interest
mailing list