[antlr-interest] unexpected char error
Martin Probst
mail at martin-probst.com
Sat Mar 24 02:06:37 PDT 2007
> But, its supposed to implicitly define the token as a literal when
> I use
> it. That's the whole point of allowing me to use the string in the
> parser. It should then appear in the literal table. IN the second
> case
> above, I see:
>
> "="=11 in my myLexerTokenTypes.txt file.
It does. But the thing is that this "test literals" feature only
tests lexed tokens for literals. Meaning, if the lexer reads
something in, and creates a token from it, it then looks up in a
table if it's one of the literals (you can switch this behaviour on/
off on a per-token type basis). If yes, it sets the type of the token
to the corresponding type.
Of course this only works if the input can be matched by any lexer
rule, and if this rule has the test literals thing turned on. This is
typically done to filter tokens like "if" from the identifier rule of
the language. I guess in your case, because identifiers are not
allowed to contain "=" tokens, it won't work.
So generally speaking this test literals feature is nice to easily
pick keywords that would also be matched by the identifier rule (or
similar rules), but it's not useful for tokens that do not match the
identifier production.
Regards,
Martin
More information about the antlr-interest
mailing list