[antlr-interest] ENHANCEMENT - Have "lexer grammar" generate recognition for string literals in tokenVocab

[Gavin Lambert]

At 15:18 10/10/2007, Austin Hastings wrote:
 >There's no loop. My current system has the parser using the
 >lexer's tokens. What I would like is for the lexer to use the
 >parser's tokens.
 >That would let the parser generate "anonymous" tokens like
 >'while'=91 and then have the lexer go "oh, a literal string. I
 >should generate an imaginary token for it with number = 91!"

But that's my point -- you can't do that without having a cyclic 
dependency.

Ok, let's imagine that what you describe is available, and you've 
got two grammar files that use it (and no output files yet).

Now you run ANTLR to compile the lexer.  Whoops, can't do that, 
don't have the parser's tokens file yet.
So you run ANTLR to compile the parser.  Whoops, can't do that 
either, don't have the lexer's tokens file yet.
Now you're stuck.

What you'd need to be able to do to resolve this is to build an 
initial lexer ignoring the vocab, then build the parser and figure 
out its tokens, then go back and build the lexer again, inserting 
the new tokens, and finally build the parser yet again since the