[antlr-interest] ANTLR C++ Lexer

[Ric Klaren]

Hi,

On 11/28/06, Maurizio de Pascale <mdepascale at dii.unisi.it> wrote:
> 1) is it possible (and how) to recover from an exception thrown by
> lexer.nextToken?

If you want to do recovery outside the lexer you could use mark/rewind
on the inputbuffer between tries (if you really want to...)

> I would like to do something like this:
>
> do
> {
>     try{
>
>     token = lexer.nextToken();
>
>     } catch (exception)
>     {
>        //spit output for user but make sure we can continue
>        lexer.resume() ?!?
>     }
> }
> while( token->getType() != Lexer::EOF_ );

If you combine an antlr parser and lexer then this scheme is probably
not the most practical. Rule of thumb keep the lexer simple (and fast)
and do your recovery in the parser where you have more context to give
more sensible error messages.

> 2) I've noticed the paraphrase option:
>
> Identifier
> options{paraphrase = "identifier";}
>
> but is there a way (and how) I can access that string from C++?
> it looks like those strings are only contained in the *TokenTypes.txt
> outputted vocabolary but not anywhere in the other C++ generated files :(
> is this a parser-only accessible stuff?

In the parser you have getTokenName() inside the lexer they're unused
if my memory serves me right.

Cheers,

Ric