[antlr-interest] unicode 16bit versus new 21bit stuff

John D. Mitchell johnm-antlr at non.net
Fri Jun 18 18:22:02 PDT 2004


>>>>> "Terence" == Terence Parr <parrt at cs.usfca.edu> writes:
[...]

> I want to do unicode "right" this time.  Anybody have a strong opinion
> about the new supplemental (beyond 16bit unicode) char values and/or
> whether 2048 is a serious token type limitation?

Do Unicode right!  I.e., yes, we need to support the "astral" planes.

Alas, 2K token types is definitely too low.


> The new system will be cool.  You'll be able to use
> Character.UnicodeBlock stuff such as vocabulary=BENGALI;

Yea!

I've always wanted to write a Klingon-based translator.  :-? :-)

Take care,
	John


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/
 



More information about the antlr-interest mailing list