[antlr-interest] ENHANCEMENT - Have "lexer grammar" generate recognition for string literals in tokenVocab

Gavin Lambert antlr at mirality.co.nz
Wed Oct 10 01:26:05 PDT 2007


Mere moments ago, I wrote:
 >What you'd need to be able to do to resolve this is to build an
 >initial lexer ignoring the vocab, then build the parser and 
figure
 >out its tokens, then go back and build the lexer again, 
inserting
 >the new tokens, and finally build the parser yet again since the 


Gah, sorry, accidentally pressed the Send key sequence.  Anyway:

... finally build the parser yet again since the lexer may have 
changed its token ids around.

All of which is quite a mess, and is what combined grammars do for 
you anyway.

 >The multiple parsers problem just doesn't happen - at least for 
me
 >- because there's one syntax parser and the rest are tree 
parsers.
 >The tree parsers depend on the output (AST) of the syntax 
parser,
 >so basically everybody want's to use the tokenVocab from the
 >syntax parser, instead of the lexer.

But you can already do that.  And if you don't have multiple 
syntax parsers then I don't see why you're not using a combined 
grammar anyway, since that would do everything you seem to want to 
do.



More information about the antlr-interest mailing list