[antlr-interest] Changing/affecting the Lexer from the Parser?

Juancarlo Añez apalala at gmail.com
Sat Nov 10 06:06:11 PST 2012


Hello, Bernard,

On Fri, Nov 9, 2012 at 7:23 PM, Bernard Kaiflin <bkaiflin.ruby at gmail.com>wrote:

> No, the lexer answers to nextToken() requests from the parser. Starting at
> the character position behind the last token consumed, it chooses the rule
> that matches the most input characters. If the input can match two rules, ANTLR
> resolves this lexical ambiguity by matching the input string to the rule
> specified first in the grammar.
>

The Python implementation of CommonTokenStream lexes all the input in one
pass storing the tokens in a list that it later indexes to deliver tokens
to the parser.

To do what I suggested, I would have to write my own token stream, and
probably resource to the "mark" family of methods to allow the parser to
backtrack.

-- 
Juancarlo *Añez*


More information about the antlr-interest mailing list