[antlr-interest] Lexer bug?

[Clifford Heath]

Gavin Lambert wrote:
> At 13:47 23/10/2007, Clifford Heath wrote:
>  >...I should recognize
>  >the string "0.12 ..  3.5" as a single token... and I'm
>  >*sure* I don't want to do that!
> True, although you could combine it with the method shown in the wiki 
> regarding how to emit multiple tokens from a single lexer rule.  Which 
> admittedly is a little messy too, but that's mostly glue code.  Once 
> that's in place the actual procedure is pretty straightforward.

And then there's the fact that the string above has white-space embedded,
which means that it potentially interacts negatively with the whitespace
handling... or maybe not in this case. Still, I already dislike that I
have to re-lex a NUMBER to find whether it's octal, hex, integer or real.
I already paid a lexer to do that for me, so why am I doing it again?

Clifford Heath.