[antlr-interest] Lexer bug? (with test cases!)

Terence Parr parrt at cs.usfca.edu
Wed Oct 24 16:02:41 PDT 2007


On Oct 24, 2007, at 7:09 PM, Gavin Lambert wrote:

thanks for the analysis.

> No; I've written a few hand-rolled parsers/lexers before, but not a  
> parser/lexer generator.  I bow to your experience there, but that  
> doesn't mean I can't say when I think there might be a "better" way  
> of doing something :)

Why not just do what lex does:

when you reach an accept state, record position in input.  Continue.   
If you get an error before next stop state, roll back  and return  
token.  Simple.  ANTLR can do same

( ('.' '0'..'9'+)=> '.' '0'..'9'+ | )

I just don't want to do that automatically.  Seriously guys. Go look  
at lex. It backtracks.  Man am i getting tired of providing a free  
education on this.

You are all correct. ANTLR doesn't automatically do what is natural.  
You can make it easily do so by mimicking lex, which backtracks.  Is  
there really anything else to say on this matter?

Ter



More information about the antlr-interest mailing list