[antlr-interest] Another parsing question

[Loring Craymer]

Randall--

Interaction between parser and lexer would only be feasible at LL(1) decision points in the parser, and that would make the resulting grammars very fragile--tweak the grammar to add an alternative (perhaps elsewhere), and that decision goes from LL(1) to LL(2).  The problem with ANTLR 3 is that Ter deliberately introduced combined grammars to mimic the appearance of scannerless parsers; I thought that that was a mistake at the time and still do.  I also find it annoying that separation of parser and lexer into different grammars is inconvenient, to say the least, and the scannerless appearance of combined grammars catches a very high percentage of newbies.

I think that what is really needed instead of parser/lexer communication is a solution to the lexer FOLLOW set problem, possibly coupled with lexer states; that would bring back the ANTLR 2 lexing capabilities which were pretty amazing.

--Loring




----- Original Message ----
> From: Randall R Schulz <rschulz at sonic.net>
> To: antlr-interest at antlr.org
> Sent: Tuesday, August 5, 2008 2:35:36 PM
> Subject: Re: [antlr-interest] Another parsing question
> 
> On Tuesday 05 August 2008 14:22, Terence Parr wrote:
> > On Aug 5, 2008, at 12:44 PM, Foust wrote:
> > >> Lexing is all done up front with no input from the parser at all.
> > >
> > > Why was that done? Is it the price you pay for infinite lookahead?
> >
> > Speed of access for ith char / token.  Will build a lazy version. :)
> 
> That's probably a good idea, but will you preserve the current 
> semantics, making the lazy version simply a performance variant, or 
> will you allow the parser to influence the behavior of the lexer, as so 
> many people initially believe to be possible when the come to ANTLR?
> 
> 
> > Ter
> 
> 
> Randall Schulz