[antlr-interest] MismatchedTokenException and how to find errors in ANTLRWorks

[Gavin Lambert]

At 08:39 13/02/2008, Matt Benson wrote:
 >Seriously, Jim:  what's the likelihood of your (or
 >anyone's) being able to sum up the reason for this
 >fairly succinctly?  What is the #1 gotcha about using
 >literals vs. tokens that ANTLR3 beginners overlook?

I gave my answer to that question yesterday (though more 
briefly).  As I see it, there are two big drawbacks to using 
literal strings in the parser:

1. The generated code becomes filled with references to "T32" and 
"T63" etc rather than anything meaningful, making it harder to 
understand.  (And also complicating things if you want to generate 
an AST later on.)

2. It's too easy to forget that each quoted string produces a 
lexer rule (which may conflict with other rules), making it harder 
to find the source of ambiguity problems.  In addition, "'x' 'y'" 
is not the same as "'xy'" in the parser, though it is in the 
lexer; this can cause confusion.