[antlr-interest] Combined grammars behave differently than separate parser/lexer grammars

[Jim Idle]

No, it is a bug and I fall for it all the time too with large grammars. It turns out to be not so easy to fix because there are cases when tokens should be added silently. I think when I looked at it, we were unable to distinguish between cases as the information needed to know what to do is elided by that point. Then Ter is working on v4 and so nothing happened with this I am afraid.

Jim

> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of Stevenson, Todd (GE Healthcare)
> Sent: Monday, May 17, 2010 10:34 AM
> To: antlr-interest at antlr.org
> Subject: [antlr-interest] Combined grammars behave differently than
> separate parser/lexer grammars
> 
> I recently split a combined grammar that I had built into separate
> parser and lexer grammars.  I noticed  that in the combined grammar
> when
> I attempt to reference an undefined token, Antlr throws an error.
> However, after I split this grammar into separate lexer and parser
> grammars, Antlr allowed me to reference lexer tokens in the parser
> grammar not defined by the lexer grammar.  These become very difficult
> to debug.   Am I missing something, or is the the correct behavior?
> 
> thanks.
> 
> 
> 
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address