[antlr-interest] MismatchedTokenException and how to find errors in ANTLRWorks

[Thomas Brandon]

On Thu, Feb 14, 2008 at 9:06 PM, Guntis Ozols <guntiso at latnet.lv> wrote:
>  > As Guntis points out, in this case it is not the literals that are
>  > the problem but the confusion arising from combined grammars which
>  > hide the lexer.
>
>  I like antlr because of combined grammars, I think it is the right direction.
>
>  And I think that the root of the problem in this case
>  is antlrs broken lexer (please, forgive me again)
Might help to elaborate on that. Given the problem currently discussed
I don't think this is broken. This is just the way all such tools
work. A lexer takes in characters and spits out tokens. A parser takes
in tokens and spits out nodes. I don't know that a tool that combined
these would be possible or a good thing and it certainly isn't what
ANTLR or any other such tool attempts to provide. I kind of agree that
combined grammars can obfuscate this point. Though I agree with Jim's
point that tools should be designed more to provide the most ease to
the people who understand them than to provide the most ease to the
people learning them (for tools like ANTLR that aren't aimed at
low-level users) so I think combined grammars are a good feature.
>  BTW I think I understand what the tool is doing in this particular
>  case and I was not asking for help, was I? Adding
>  System.err.println("Anonymous token in parser rule: " + lit);
>  to AssignTokenTypesWalker.assignStringTypes(Grammar) works well for me.
But surely for those that understand and want to use parser defined
literals having many such messages printed out is just unacceptable.
Either remove them or add an option for them, a message about them
doesn't seem a good compromise.

Tom.
>
>  Guntis
>
>