[antlr-interest] Disabling recovery during parsing
Terence Parr
parrt at cs.usfca.edu
Thu Jan 10 13:32:45 PST 2008
On Nov 8, 2007, at 2:42 PM, Curtis Clauson wrote:
> The entire error recovery and reporting systems are not
> appropriately documented, and your frustration is well understood.
>
> Both the lexer and parser handle error recovery similarly, but with
> different method calls.
>
> The lexer will call
> recover(re)
> before it throws a RecognitionException, which consumes the
> unexpected character. The top
> nextToken()
> method has a hard-coded exception handler that catches the
> RecognitionException, reports the error, and calls recover(re) again
> (a bug in my opinion).
Excellent catch, Curtis. Fixed by only consuming if not consume
previously:
catch (NoViableAltException nva) {
reportError(nva);
recover(nva); // throw out current char and try again
}
catch (RecognitionException re) {
reportError(re);
// match() routine has already called recover()
}
fixed:
http://www.antlr.org:8888/browse/ANTLR-209
> One important, and also undocumented, note: CommonTokenStream will
> lex the *entire* stream into tokens on the first token fetch. This
> means the lexer will process and display all errors before the
> parser processes the first token. So much for context.
Ah. I thought I had an unbuffered token buf; somebody else might have
sent one to list. Anyhoo, i only have UnBufferedTreeNodeStream at the
moment.
> When the parser fails to match a token, it calls
> mismatch(...)
> which creates the exception object and then calls
> recoverFromMismatchedToken(...)
> This method looks ahead at the next token and if it matches, reports
> the error, skips the unexpected token, and returns a successful
> match. If it does not match, it calls
> recoverFromMismatchedElement(...)
> This method tests if the unexpected token could follow the expected
> token. If so, it will report the error, and return a successful
> match (acting as if the missing token were found). If not, the
> exception object is finally thrown.
>
> If you have not created your own rule exception handler in the
> grammar, or configured the default exception handlers with
> @rulecatch {}, then the exception will be caught by the default rule
> exception handler, which will call
> reportError(re)
> recover(input, re)
>
> The recover(...) method in the parser will try to consume tokens
> until one is found that allows it to resynchronize and continue
> parsing the rest of the tokens.
>
> To change how recovery is handled in the parser, override
> recoverFromMismatchedToken(...)
> recoverFromMismatchedElement(...)
> to change those strategies, or override
> mismatch(...)
> to change the whole before-exception response.
> To change how recovery is done in response to the RecoveryException,
> configure or provide a different either default or per-rule
> exception handler in the grammar, or override
> reportError(re)
> recover(input, re)
> in the parser.
>
> I had to spelunk the source to find all of this, since these
> questions never get answered. I has worked for me so far.
Thanks for digging around in there. code is doc ;) Also, i'm pretty
sure a lot of this is covered in book on how to override error message
etc...
Ter
More information about the antlr-interest
mailing list