[antlr-interest] Re: lexer "modes" for XML parsing etc...

[Micheal J]

> Your wish is my command.  ;)  Do we need something like
> 
> lexer grammar L;
> 
> ID : ... ;
> SQLSTART : "sql(" {pushContext(SQL);} ;
> WS : ... ;
> 
> context SQL {
> ID : ... ;
> ACTION : ...;
> STRING : ... ;
> ENDSQL : ')' {popContext();}
> }
> 
> context island2 {
> ...
> }
> 
> [note the push/pop rather than simple set; very useful]
> 
> Then, the lexer would simply generate multiple Tokens-like rules for  
> all contexts?  You see a different lexer entry rule for each  
> context.  How do you switch?  We'd need an int constant (as we have  
> no function poitners in Java--a pox on their family) that would jump  
> to the right starting method.
> 
> Sounds easy.  Is this what we want?  It is proper for island 
> grammars  
> that feed off the same input stream.  Multiple input streams like  
> include files need to be handled with a multiplexing input buffer.


How is this different from using TokenStreamSelectors?. Is this just a
ANTLR-generates-the-tokenselector-plumbing-and-glue-too scenario. Or is this
a case where all the switching in done within a single lexer with no
tokenselector-style switching?

Cheers,

Micheal