[antlr-interest] Disabling recovery during parsing

[Terence Parr]

On Nov 8, 2007, at 2:42 PM, Curtis Clauson wrote:

> The entire error recovery and reporting systems are not  
> appropriately documented, and your frustration is well understood.
>
> Both the lexer and parser handle error recovery similarly, but with  
> different method calls.
>
> The lexer will call
>  recover(re)
> before it throws a RecognitionException, which consumes the  
> unexpected character. The top
>  nextToken()
> method has a hard-coded exception handler that catches the  
> RecognitionException, reports the error, and calls recover(re) again  
> (a bug in my opinion).

Excellent catch, Curtis.  Fixed by only consuming if not consume  
previously:

			catch (NoViableAltException nva) {
				reportError(nva);
				recover(nva); // throw out current char and try again
			}
			catch (RecognitionException re) {
				reportError(re);
				// match() routine has already called recover()
			}

fixed:

http://www.antlr.org:8888/browse/ANTLR-209


> One important, and also undocumented, note: CommonTokenStream will  
> lex the *entire* stream into tokens on the first token fetch. This  
> means the lexer will process and display all errors before the  
> parser processes the first token. So much for context.

Ah. I thought I had an unbuffered token buf; somebody else might have  
sent one to list.  Anyhoo, i only have UnBufferedTreeNodeStream at the  
moment.

> When the parser fails to match a token, it calls
>  mismatch(...)
> which creates the exception object and then calls
>  recoverFromMismatchedToken(...)
> This method looks ahead at the next token and if it matches, reports  
> the error, skips the unexpected token, and returns a successful  
> match. If it does not match, it calls
>  recoverFromMismatchedElement(...)
> This method tests if the unexpected token could follow the expected  
> token. If so, it will report the error, and return a successful  
> match (acting as if the missing token were found). If not, the  
> exception object is finally thrown.
>
> If you have not created your own rule exception handler in the  
> grammar, or configured the default exception handlers with  
> @rulecatch {}, then the exception will be caught by the default rule  
> exception handler, which will call
>  reportError(re)
>  recover(input, re)
>
> The recover(...) method in the parser will try to consume tokens  
> until one is found that allows it to resynchronize and continue  
> parsing the rest of the tokens.
>
> To change how recovery is handled in the parser, override
>  recoverFromMismatchedToken(...)
>  recoverFromMismatchedElement(...)
> to change those strategies, or override
>  mismatch(...)
> to change the whole before-exception response.
> To change how recovery is done in response to the RecoveryException,  
> configure or provide a different either default or per-rule  
> exception handler in the grammar, or override
>  reportError(re)
>  recover(input, re)
> in the parser.
>
> I had to spelunk the source to find all of this, since these  
> questions never get answered. I has worked for me so far.

Thanks for digging around in there.  code is doc ;)  Also, i'm pretty  
sure a lot of this is covered in book on how to override error message  
etc...

Ter