[antlr-interest] ENHANCEMENT - Have "lexer grammar" generate recognition for string literals in tokenVocab
Gavin Lambert
antlr at mirality.co.nz
Tue Oct 9 13:24:04 PDT 2007
At 02:31 10/10/2007, Austin Hastings wrote:
>
>When more than one parser is in use, they need to share a common
>(initial) vocabulary. This can come from the source lexer via
the
>tokenVocab option.
[...]
>I would like the "lexer grammar" mode of antlr to be extended to
>accept a tokenVocab (it currently does this) and further to
>generate the appropriate tokens for string literals it finds in
>the tokenVocab.
I can't see how that could possibly work -- you've set up a cyclic
dependency. You can't build the parser until you've built the
lexer (since the parser uses the lexer's tokens) and you can't
build the lexer until you've built the parser (since the lexer
uses the parser's extra tokens). So you can't build anything
unless you've already built it, which doesn't work.
(The problem is even worse when you have multiple parsers and a
common lexer, which is what you appear to be describing -- the
lexer has to support the union of string literals from all
potential parsers, or they won't work.)
Just define your string literals in a "tokens" block at the top of
the lexer. That's what it's for.
More information about the antlr-interest
mailing list