[antlr-interest] Re: lexer "modes" for XML parsing etc...

[Micheal J]

> > How is this different from using TokenStreamSelectors?. Is 
> this just a 
> > ANTLR-generates-the-tokenselector-plumbing-and-glue-too scenario.
> > Or is this
> > a case where all the switching in done within a single lexer with no
> > tokenselector-style switching?
> 
> Well you need the multiplexor thing for multiple input streams and  
> the same sort of selector thing works for multiple lexers 
> feeding off  
> the same input stream.  THis would only be "sugar" to make it  
> easier.  currently you have to open a new file, add the new lexer  
> bits (possibly having complications due to instance variables/ 
> methods), run antlr on it, and add some glue during the parser/lexer  
> construction and launching.

OK. I was trying to contrast it with an all-in-one hand-coded
lexer+preprocessor Kunle & I built where there is a int to store the current
state (or lexing context/mode if you will) and, GetNextToken() switch()-es
on the current_lexer_state_int to determine which of a number of
NextToken_XXX() methods to delegate to.

This sounds like what we're now planning for ANTLR 3. Is it?

Cheers,

Micheal