[antlr-interest] greedy vs nongreedy lexer rules

[Terence Parr]

Actually v3 by default no longer slurps of the entire input. you have to use the buffered version :)

We definitely don't like interaction between the parser and lexer because lookahead makes it difficult to determine when you do things. best to keep them totally separate. That said, the lexer can do a lot of state tracking itself if necessary.

Ter
On Apr 18, 2010, at 6:03 PM, Graham Wideman wrote:

> Quick question:
> 
> I noticed Ter say this:
> 
> At 4/18/2010 04:56 PM, you wrote:
>> Yeah, best we can do is linear for NFA evaluation given infinite processors, though, unless I allow lexer to race ahead of parser, buffering up all the tokens. 
> 
> ... was there a decision made at some point to have the lexer not slurp the entire input at once like v3?   If so I'd like to read more about pros and cons of that, and whether that facilitates more interactivity between lexer and parser... was there a thread on that at some point?
> 
> Thanks,
> 
> Graham 
> 
> 
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address