[antlr-interest] Lexer code not generated as expected?

Gavin Lambert antlr at mirality.co.nz
Tue Dec 15 11:41:21 PST 2009

At 08:25 16/12/2009, Peter Boughton wrote:
 >The ANTLR book mentions auto-backtracking (which seems to be 
 >is *not* happening here), and which can be turned on with 
 >Would that not solve this problem?

Unfortunately not; backtracking is a parser feature and doesn't 
operate in the lexer.

 >If so, are there any issues (aside from performance) that might 
 >a reason not to have this turned on?

The biggest problem with it (other than performance) is that it 
can hide errors in the grammar, and lead to misleading error 
messages while processing input (since when it finally does 
produce an error, it's likely to be off in some weird part of the 

But yes, it would be nice if lexing ambiguity could be dealt with 
a little more automatically.  But at the end of the day, fixing it 
up yourself will improve performance, so it's not all bad news.

More information about the antlr-interest mailing list