[antlr-interest] unicode 16bit versus new 21bit stuff
John D. Mitchell
johnm-antlr at non.net
Fri Jun 18 18:22:02 PDT 2004
>>>>> "Terence" == Terence Parr <parrt at cs.usfca.edu> writes:
[...]
> I want to do unicode "right" this time. Anybody have a strong opinion
> about the new supplemental (beyond 16bit unicode) char values and/or
> whether 2048 is a serious token type limitation?
Do Unicode right! I.e., yes, we need to support the "astral" planes.
Alas, 2K token types is definitely too low.
> The new system will be cool. You'll be able to use
> Character.UnicodeBlock stuff such as vocabulary=BENGALI;
Yea!
I've always wanted to write a Klingon-based translator. :-? :-)
Take care,
John
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/antlr-interest/
<*> To unsubscribe from this group, send an email to:
antlr-interest-unsubscribe at yahoogroups.com
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list