[antlr-interest] Modifying lexer state from parser

Ric Klaren ric.klaren at gmail.com
Mon Feb 12 08:40:07 PST 2007


Hi,

On 2/12/07, Omry Yadan <omry at yadan.net> wrote:
> I want my lexer to start skipping whitespaces after a certain condition
> is met in the Parser.
>
> what I am trying to do is to access the lexer and modify it when this
> condition is met.
>
> something like this (note the magical $lexer.m_conditionMet which is
> just wishful thinking):

Yes you can (like Yue and Prashant said)....

But... in general you *really* have to know what you're doing. Because
generally speaking the lexer can be way ahead of the parser due to
guessing mode (syntactic predicates) which will lead to really strange
behaviour e.g. it will start skipping spaces in another spot than you
expect.

If you're parser is LL(1) and does not use syntactic predicates (the
'=>' operator) then it is safe (I think). If that is not the case then
it *might* be safe, but you really have to know how antlr generates
code and how your parser behaves on input.

Cheers,

Ric


More information about the antlr-interest mailing list