[antlr-interest] Help controlling parser decisions
Randall R Schulz
rschulz at sonic.net
Tue Jul 24 11:04:36 PDT 2007
On Tuesday 24 July 2007 10:56, Ted Villalba wrote:
> Hi,
>
> I have a grammar that contains tokens that are sometimes operators,
> sometimes not, depending on the context. The set of operators
> overlaps with the set of all words that can be acceptable tokens.
> Trouble is, depending on the order of my lexer rules, the parser
> will recognize all such tokens (AND , OR ,NEAR) as operators, or will
> recognize none of them as operators.
This qualifies as a FAQ, I think.
Lexical analysis is not driven by parser rules. It is an entirely
separate, earlier phase of input processing. There is no way to produce
different token types from any given input depending on context.
Another way of thinking about it is that while the parser is driven by
top-down invocation of rules (along with all the look-ahead and
backtracking that may be involved), there is no such orchestration of
the lexical analyzer. It just chunks up the characters (or breaks down
the total sequence) into a linear chain of sub-sequences of the total
input.
> ...
>
> Ted
Randall Schulz
More information about the antlr-interest
mailing list