[antlr-interest] Capturing lexer exceptions
Gerz, Michael
Gerz at fgan.de
Wed Nov 1 22:11:29 PST 2006
Dear Jim,
thank you very much!
What you propose (adding an "ELSE" rule to the lexer) sounds like a great idea. I will have a look at this solution.
Best regards,
Michael
************************************************************************
Dr. Michael Gerz
FGAN e.V. Phone: +49 228 9435-414
Department FKIE/ITF Fax: +49 228 9435-685
Neuenahrer Straße 20 E-Mail: gerz at fgan.de
53343 Wachtberg-Werthhoven WWW: www.fgan.de
> -----Ursprüngliche Nachricht-----
> Von: Jim Idle [mailto:jimi at intersystems.com]
> Gesendet: Dienstag, 31. Oktober 2006 21:44
> An: Gerz, Michael; antlr-interest at antlr.org
> Cc: Schueller, Henriette
> Betreff: RE: [antlr-interest] Capturing lexer exceptions
>
> Michael,
>
> I think that really, you should not have a lexer that can
> generate errors, in that it should be able to handle anything
> thrown at it and produce some sort of token or handle it in
> some way. Hence the ANTLR3 lexer just reports the character
> it sees that it doesn't know what to do with and throws it
> away (consumes it).
>
> So, if your lexer spec does not cover all the characters that
> might come in to it, then this is what will happen. However,
> you can probably achieve what you are looking for by specifying:
>
> UNKNOWN: . {do something, or not}
>
> as the last lexer rule, and either looking in the parser for
> this and reporting the error in some way yourself, or just
> setting the token to be skipped.
>
> You might also find the filter mode is what you are looking
> for in the lexer? Check the examples for a filtering lexer.
>
> Yet another option is to override the lexer error handler, or
> more specifically error display using @lexer:: ... and just
> don't print anything, which should cause the character to be
> consumes silently.
>
> However, my own opinion is that you should make sure your
> lexer is able to process all characters and perform some
> specific, deliberate action such as set the token to be
> skipped as you then comment it as to why you are doing this
> and it just feels more logical to me, unless you are
> specifically constructing some sort of filter.
>
> Jim
More information about the antlr-interest
mailing list