[antlr-interest] Lexer problem ( add ALLCHARS )

Martin Probst mail at martin-probst.com
Thu Mar 8 03:13:43 PST 2007


Hi,

from what I see the input "aa" can be matched by these rules in your  
lexer grammar:
IDENTIFIER, RANGED, ARBITRARYLITERAL, ALLCHARS
So the lexer cannot figure out which rule to use for this input, and  
apparently it picks one you don't expect. You should get lots of  
warning by ANTLR about this, did you ignore those? You certainly  
shouldn't!

You will have to make up your mind on what literal should be selected  
at which point of time. It's usually better to resolve such issues in  
the parser.

Martin

PS: can someone change ANTLR so that it includes the type of the  
mismatched token in the error message? I think this is one of the  
most common problems, and users would at least have a hint on what's  
going on if they'd see "expecting IDENTIFIER, found 'aa' (ALLCHARS)".


More information about the antlr-interest mailing list