[antlr-interest] MismatchedTokenException and how to find errors in ANTLRWorks

[Loring Craymer]

I've learned that "the customer is always right" is good policy on bug reports--not that he is right about the specifics, but that he is usually right that "there is a problem".  As Guntis points out, in this case it is not the literals that are the problem but the confusion arising from combined grammars which hide the lexer.  I argued with Ter against having combined grammars for that reason; where ANTLR historically has differed from other "upscale" translation packages is in its "one grammar, one pass" approach, especially for tree grammars.  However, Ter and I agreed to disagree on this issue; the result is that support for combined grammars should include more detailed documentation on the subject of literals and their use.   Jim's approach of discouraging beginners from using them has to be incorrect; literals are not an advanced feature but a user convenience if they are properly documented.

--Loring

----- Original Message ----
> From: shmuel siegel <antlr at shmuelhome.mine.nu>
> To: antlr-interest at antlr.org
> Sent: Wednesday, February 13, 2008 10:31:38 AM
> Subject: Re: [antlr-interest] MismatchedTokenException and how to find errors in ANTLRWorks
> 
> Jim Idle wrote:
> > ....
> >
> > There isn't anything wrong with the tool, it is doing what you tell it 
> > to do. The token isnt anonymous, it is called T66 or whatever, the 
> > runtime doesn't know that you created it by specifying it in the parser. 
> > So, I don't see why "anonymous token" would help you at all.
> >
> > .....
> >> The tool needs to be patched, not users.
> >>     
> >
> > A tool should strive to be as simple as possible, but no simpler :-) 
> > While that statement makes sense, you are an under an obligation to 
> > yourself to try to make sure you understand the tool too. Ironically, my 
> > statement that literals in parsers are confusing is based entirely upon 
> > your stated premise.
> >
> > Jim
> I thought that another voice should jump in here. Even though Jim and I 
> are usually on different sides of usability arguments (Jim takes the 
> side of the ANTLR developer and I take the side of the ANTLR rule 
> writer), I totally agree with him here. Not only is ANTLR doing what you 
> told it to do, it is doing what you would want it to do if you were more 
> experienced with the tool. Personally, I am amazed that ANTLR can 
> distinguish between '+', '=', and '+='. Criticism of the tool should be 
> postponed until you truly understand what the tool is doing.
> ANTLR is a parser and lexer generator (okay, it is other things too). 
> The fact that ANTLR lets you implicitly generate lexer rules by using 
> literals in the parser does not change the fact that they are lexer 
> rules and must conform to the lexer paradigm.
> 
> Shmuel
> 
> 




      ____________________________________________________________________________________
Be a better friend, newshound, and 
know-it-all with Yahoo! Mobile.  Try it now.  http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ