[antlr-interest] Re: lexer "modes" for XML parsing etc...

[Terence Parr]

On Nov 19, 2005, at 1:19 PM, Oliver Zeigermann wrote:

> Hi Terence,
>
> I suppose something like this will work as a quick hack, but you do
> not consider this a permanent fix, right. Because it's ugly...

Mainly thinking of quick fix, but we can formalize to get you the  
kind of predicates you want.

> What you describe for v3 final really sounds like lexer modes which -
> as far as I remember - you don't like?! Finally giving in? Hihihihi ;)

I'm thinking of it as more of a goal-oriented lexer with multiple  
contexts.  Note that it's still better than modes in lex which people  
use to match strings etc...  You should use a separate rule in ANTLR  
for that.  For context-sensitive lexical regions in a single input  
stream, however, context-sensitive lexers can be implemented in a  
modal sense; here the mode implies context which I think is probably  
ok. ;)  I won't call it a mode though ;)

It will be very nice eventually when the parser can say "heh, I'm  
looking for an INT or an ID (and that's it)" so that messages are  
really good and the context is so specific it can built a smaller  
faster predictor.  This solves the List<List<int>> issue of the ">>"  
as it will only be looking for a '>' :)

Ter