[antlr-interest] Understanding priorities in lexing (newbie)

Micheal J open.zone at virgin.net
Thu Jul 12 16:07:38 PDT 2007


Hi,

> > The point is that 'ab' *doesn't* match KEYWORD -- except in the
> > mind of the predictor, since it isn't checking the whole rule.  So  
> > an input of 'ab' should unambigously result in OTHER OTHER; an  
> > input of 'abc' *could* result in either OTHER OTHER OTHER or  
> > KEYWORD, but the normal "pick the longest match and/or the first  
> > listed" rules sort out that ambiguity.
> 
> Yes.  ANTLR doesn't do the natural thing here.  For normal cases,  
> it's not an issue.  Few tokens are prefixes like that.  
> Normally it's  
> keyword against 'a'..'z'+ not 'a'..'z'.

What would it take to alter the prediction strategy/algo to do the "natural
thing" even in the presence of token prefixes like these?

Just trying to get an idea of the work involved. I can see how this might be
an issue to some.

Micheal


-----------------------
The best way to contact me is via the list/forum. My time is very limited.



More information about the antlr-interest mailing list