[antlr-interest] MismatchedTokenException and how to find errors in ANTLRWorks
Guntis Ozols
guntiso at latnet.lv
Tue Feb 12 15:58:27 PST 2008
> >Seriously, Jim: what's the likelihood of your (or
> >anyone's) being able to sum up the reason for this
> >fairly succinctly? What is the #1 gotcha about using
> >literals vs. tokens that ANTLR3 beginners overlook?
>
> I gave my answer to that question yesterday (though more
> briefly). As I see it, there are two big drawbacks to using
> literal strings in the parser:
>
> 1. The generated code becomes filled with references to "T32" and
> "T63" etc rather than anything meaningful, making it harder to
> understand. (And also complicating things if you want to generate
> an AST later on.)
>
> 2. It's too easy to forget that each quoted string produces a
> lexer rule (which may conflict with other rules), making it harder
> to find the source of ambiguity problems. In addition, "'x' 'y'"
> is not the same as "'xy'" in the parser, though it is in the
> lexer; this can cause confusion.
Sure it is confusing. I fell into this 'xy' trap, too.
If COLON is defined and STAR is defined, then creating
another token for ':*' in a *parser* rule is confusing.
It should be fixed in antlr.
At least, antlr should emit 'anonymous token' warning[s].
I prefer literals because of simplicity / readability.
The tool needs to be patched, not users.
Guntis
More information about the antlr-interest
mailing list