[antlr-interest] Tokens vs. Characters in Lexer/MismatchedTokenException

[Rick Mann]

On Jan 18, 2009, at 13:12:30, Terence Parr wrote:

> ah.  Yes, I get it now. Take of a lexer as a parser that parses
> characters instead of tokens. In this way I have generalized the
> notion of a recognizer so that we represent any element in the stream
> as an integer vocabulary symbol "type".
>
> sorry for the confusion.


No worries. The use of the Java primitive type "int" threw me a bit,  
is all. I've typedef'd a "CharTokenT" to use, might find a better name  
later. I guess this is okay because you'll only ever be looking at  
characters or token types exclusively, never a mix.

Thanks!

-- 
Rick