[antlr-interest] Tokenising for context specific reserved words
Terence Parr
parrt at cs.usfca.edu
Fri Jul 18 13:58:00 PDT 2008
On Jul 18, 2008, at 1:59 AM, Loring Craymer wrote:
> Actually, I just thought of a hybrid approach: support subtypes for
> double-quoted literals and have the subtype field set by the lexer.
> Then, a doubly quoted literal reference in a grammar is handled by
> looking at the subtype field, while an ID reference looks at the
> type field. Except for the extra field (which probably costs
> nothing--minimum allocation sizes are usually larger than tokens),
> the only added runtime overhead would be in the lexer to set the
> subtype field.
There was an interesting paper called "Schrodinger's tokens"... if you
are a physics or quantum physics buff, you get the reference ;)
Ter
More information about the antlr-interest
mailing list