[antlr-interest] Lexer strangeness

[Clifford Heath]

Austin Hastings wrote:
> Before you get too involved in this problem, consider your statement,
> "IDENTIFIER may not end with a digit, but VARIABLE might."
> That right there tells me you are going to have grammar problems later, 
> because you will never completely trust the lexer to determine 
> IDENTIFIERs from VARIABLES - after all, what is "foo"?
> 
> You should probably establish a firm rule - variables always end in a 
> digit - to simplify things. Or handle disambiguation in the parser, 
> rather than in the lexer. (My favorite.)

You're quite right, of course, and that's exactly what I had just done :-).

Clifford Heath.