[antlr-interest] Lexer exception?

Jens Bartelheimer jens.bartelheimer at gmx.de
Mon Feb 20 23:47:54 PST 2006


Thanks for your help. Your are right. It is an error raised by the 
parser. And if I set the option defaultErrorHandler=false in the options 
of the parser I can catch the exception. Thanks, Jens

Alexey Demakov wrote:
> As far as I know defaultErrorHandler = false causes lexer
> to throw TokenStreamException. By default parser catches 
> RecognitionException only,
> so parser will pass TokenStreamException without any messages.
>
> I think
>
> line 1:1: unexpected token: 111
>
> caused by error in parser not in lexer. It looks like default parser 
> error message.
> If 111 is valid token, there is no error in lexer. If file can't start 
> from
> this token, it is parser error.
>
> Regards,
> Alexey
>
> -----
> Alexey Demakov
> TreeDL: Tree Description Language: http://treedl.sourceforge.net
> RedVerst Group: http://www.unitesk.com
>
>
> ----- Original Message ----- From: "Jens Bartelheimer" 
> <jens.bartelheimer at gmx.de>
> To: "Alexey Demakov" <demakov at ispras.ru>
> Cc: "antlr-interest" <antlr-interest at antlr.org>
> Sent: Monday, February 20, 2006 12:48 PM
> Subject: Re: [antlr-interest] Lexer exception?
>
>
>> Ok, this method solves the problem if the option is set to true.
>>
>> As Martin mentioned I have to set this option to false but there are 
>> no additional exceptions. I think this is idependent from the bug in 
>> antlr. Am I right? I also do not want to skip the malformed token, I 
>> only need an exception so that I can recognize the wrong input  if  I 
>> call the parser. Further I do not need the printing of the message 
>> "line 1:1: unexpected token: 111". Is this only possible with the 
>> $skipToken....?
>>
>> Thanks,
>> Jens
>>
>> Alexey Demakov wrote:
>>> Hi Jens,
>>>
>>> Aa I wrote recently, it is the bug in antlr.
>>> http://www.antlr.org/pipermail/antlr-interest/2006-February/015417.html 
>>>
>>>
>>> Define this method in your lexer code:
>>>
>>> public void recover( RecognitionException ex, BitSet tokenSet ) 
>>> throws CharStreamException {
>>>  saveConsumedInput = false;  consume();
>>>  consumeUntil( tokenSet );
>>>  saveConsumedInput = true; }
>>>
>>> But if you want completly skip malformed token, you have to define 
>>> custom exception handler
>>> with $setType( Token.SKIP ) for each lexer rule. :(
>>>
>>> ---
>>> Alexey Demakov
>>> TreeDL: Tree Description Language: http://treedl.sourceforge.net
>>> RedVerst Group: http://www.unitesk.com
>>>
>>>
>>> From: "Jens Bartelheimer" <jens.bartelheimer at gmx.de>
>>>> Hi Martin,
>>>>
>>>> thanks for your reply but if I turn off the option there are no 
>>>> additional exceptions.
>>>>
>>>> If I turn it on explicit I have the following error in the Lexer:
>>>>
>>>> My error message:
>>>> The method recover(RecognitionException, BitSet) is undefined for 
>>>> the type UMLLexer
>>>>
>>>> Why? Is there a mistake in my Lexer?
>>>>
>>>> Jens
>>>>
>>>>
>>>> Martin Probst wrote:
>>>> >> how can I catch such an exception? I do not found something in 
>>>> the >> documentation.
>>>> >>
>>>> >> line 1:1: unexpected token: 111
>>>> >>     >
>>>> > You have to turn off the default error handler, e.g. in your options
>>>> > section of the lexer "defaultErrorHandler = false". Afterwards 
>>>> you can
>>>> > catch exceptions. All of them extend ANTLRException, the manual 
>>>> contains
>>>> > a part called "Exception hierarchy".
>>>
>>>
>>>
>
>


More information about the antlr-interest mailing list