[antlr-interest] ANTLR or ANTLR Eclipse plugin bug

[Prashant Deva]

Hi Martin,

On 8/9/05, Martin Olsson <mnemo at minimum.se> wrote:
> Hi,
> 
> I've been trying for quite a while now to get my lexer to handle unknown
> chars (report error, ignore char, resume lexing for new token). I've now
> managed to implement something that achieves this but I had to resort to a
> little semi-ugly hack.
> 
> For the record, I'm using the latest version of the ANTLR plugin for
> Eclipse. Atleast when I use the Eclipse update site feature it says I
> already got the latest version. I couldn't really tell exactly which
> version this is, but this is what the version info in Eclipse look like
> currently:
> 
>   http://mnemo.minimum.se/screens/version_box.png
> 
> Anyway, back to the matter at hand. From the ANTLR docs, it seemed like
> the default error handling would do exactly what I needed. Which I assume
> also means that defaultErrorHandler is implicitly set to true when not
> specified, correct? Anyway, I looked at the lexer source and saw that the
> lexer did NOT do what the docs said, instead it did raise an exception
> which when propagated to the parser would cause the entire lex/parse
> process to stop.
The lexer is supposed to do that. Read the doc on error handling. The
lexer needs to inform the parser when something gets screwed up in the
input stream.

> So I tried to set defaultErrorHandler to true in the
> lexer options, generated the lexer and parser and boom, now the lexer was
> full of errors. It said "missing method recover()". Now the error handling
> started to look more like what I actually wanted, it reported and error
> and then called recover() just like in the parser. So my hack was to add a
> dummy method called recover() with the right params, and then have this
> method so nothing at all. This silenced the errors and the code works, but
> it feels rather ugly.
This has nothing to do with the antlr eclipse plugin.
It seems to be a bug in the code gen. The recover() method is used
inside the parser.
The lexer as such i dont think can recover the lexing of the same
token. if something screws up, it will just skip that char to find the
next token.

Seeing this I believe the defaultErrorHandler option should be
disallowed for lexer rules.

PRASHANT

> 
> Unfortunately my current code is closed-source proprietary so I cannot
> attach the .g file here even though I understand that this would help
> reproduce the error etc.
> 
> So, the big question is; why does ANTLR generate broken code? Is this
> fixed in cvs? Or, is it not an ANTLR problem at all? Maybe it's a problem
> with the Eclipse ANTLR plugin?
> 
> I've tried to contact the ANTLR Eclipse plugin author too (hi again if
> you're on this list too ;D).
> 
> What do you make of this?
> 
> 
> 
> 
> Regards,
> martin
>