[antlr-interest] Re: Real examples of error recovery

[Ric Klaren]

On Tue, Aug 24, 2004 at 04:00:59PM -0000, Paul J. Lucas wrote:
> --- In antlr-interest at yahoogroups.com, Richard Clark <rd_clark at s...> wrote:
> > Paul, can you provide some examples of where you think the lexer would
> > need to backtrack?
>
> I never said it needed to backtrack.  I said it needed to change state.
> Look at:
>
> http://www.w3.org/TR/xquery/#id-lexical-states

I assume you're using multiple lexers and a tokenstream selector to
implement those?

> (I'm making this example up as I go, but hopefully, you get the idea.)
> Suppose you start in the default state.  Now you get:
>
> declare collation "foo";
>
> The "keyword" of "default" is missing.  Hence the lexer never switches to
> the NAMESPACEDECL state.  Before I lex "foo" I need to poke the lexer and
> be able to forceably switch its state to NAMESPACEDECL.  Or make the lexer
> skip forward to just past the ';' and reset its state.  Or something.
>
> FYI: because all of the complexity is in the lexer, it makes writing the
> grammar for the parser be as if XQuery *does* have keywords, i.e., you
> write the grammar "normally," e.g.:
>
> defaultCollationDecl : DECLARE DEFAULT COLLATION uri=stringLiteral ;
>
> How would I annotate the above grammar rule with an "exception/catch" and
> make a best effort to continue?

Tack something like this after the rule:

rule: .... ;
exception catch [antlr::RecognitionException &ex]  { }

Best effort to continue for that you'd have to analyze the grammar.
Basically you can use the various consumeXYZ methods of the lexer to skip
ahead to parts. You could even call a separate parser there to do 'smart'
things. Error recovery can be done by advancing tokens or by inserting
tokens (the latter could be done by adding a tokenstream between the lexer
and the parser)

If you catch the exception at the end a rule and handle it there then
parsing continues at the calling rule. (after the call to current rule) So
if you can fix the input stream to be sane for that point then things will
kinda work. You can use the $FOLLOW/$FIRST sets inside the exception
handler or $FOLLOW(rule) $FIRST(rule) to do calculations outside of a rule.

There's probably a load of papers available on the net on recovery
strategies.

>  How to tell the lexer to do something like "skip until you find a ';'"?

consumeUntil(); Check out the CharScanner docs/class definition.

Cheers,

Ric
--
-----+++++*****************************************************+++++++++-------
    ---- Ric Klaren ----- j.klaren at utwente.nl ----- +31 53 4893755  ----
-----+++++*****************************************************+++++++++-------
  "You can't expect to wield supreme executive power just because some
   watery tot throws a sword at you!"
  --- Monty Python and the Holy Grail



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
    antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/