[antlr-interest] Re: lexer "modes" for XML parsing etc...

[Terence Parr]

On Nov 19, 2005, at 1:30 PM, Oliver Zeigermann wrote:
>> It will be very nice eventually when the parser can say "heh, I'm
>> looking for an INT or an ID (and that's it)" so that messages are
>> really good and the context is so specific it can built a smaller
>> faster predictor.  This solves the List<List<int>> issue of the ">>"
>> as it will only be looking for a '>' :)
>
> Oh, so I did not get the point. But, is it really possible to only ask
> for a certain set of possible tokens "dynamically"? I mean, the lexer
> does not know what the parser will ask for at generation time.

Correct.  The parser would not call Tokens rule it would call a  
different start rule for each unique context in the grammar.  Well,  
unique in the sense that the set of tokens is different.

If you have a rule:

a : A
   | B
   | C
   ;

Rule 'a' might be able to tell the lexer to get one of A,B,C rather  
than any token.  I just don't know how lookahead would work in this  
environment.

Ter