[antlr-interest] Re: Problems with Unicode support in ANTLR
micheal_jor
open.zone at virgin.net
Thu May 16 20:10:46 PDT 2002
> Perhaps I am missing the thrust of this exchange, but
> my requirement for Unicode in Antlr is to support an English
command set and
> foreign language (initially Japanese)
> variable names.
> This talk of character blocks seems too restrictive.
> I need full Unicode support to cover the possible variable names.
ANTLR allows you to set the charVocabulary to include almost all
Unicode character values (the exceptions being \uFFFF - used for EOF -
and perhaps \u0000..\u0002).
The discussion is about making ANTLR smarter so that it understands
standard character groupings so that ANTLR .g files may be more
compact (and understandable). There might also be a performance
benefit (or deficit).
This is very important IMO. I have included part of the definition
required for a sample Unicode category if done manually in an ANTLR
definition file. It is auto-generated by a tool I wrote and may
contain errors ;-)
I believe it illustrates the value of these proposals quite nicely.
Micheal
==========================================================
Unicode Category or Class: Lu
| '\u0041'..'\u005A'
| '\u00C0'..'\u00D6'
| '\u00D8'..'\u00DE'
| '\u0100'
| '\u0102'
| '\u0104'
| '\u0106'
| '\u0108'
| '\u010A'
| '\u010C'
| '\u010E'
| '\u0110'
| '\u0112'
| '\u0114'
| '\u0116'
| '\u0118'
| '\u011A'
| '\u011C'
| '\u011E'
| '\u0120'
| '\u0122'
| '\u0124'
| '\u0126'
| '\u0128'
| '\u012A'
| '\u012C'
| '\u012E'
| '\u0130'
| '\u0132'
| '\u0134'
| '\u0136'
| '\u0139'
| '\u013B'
| '\u013D'
| '\u013F'
| '\u0141'
| '\u0143'
| '\u0145'
| '\u0147'
| '\u014A'
| '\u014C'
| '\u014E'
| '\u0150'
| '\u0152'
| '\u0154'
| '\u0156'
| '\u0158'
| '\u015A'
| '\u015C'
| '\u015E'
| '\u0160'
| '\u0162'
| '\u0164'
| '\u0166'
| '\u0168'
| '\u016A'
| '\u016C'
| '\u016E'
| '\u0170'
| '\u0172'
| '\u0174'
| '\u0176'
| '\u0178'..'\u0179'
| '\u017B'
| '\u017D'
| '\u0181'..'\u0182'
| '\u0184'
| '\u0186'..'\u0187'
| '\u0189'..'\u018B'
| '\u018E'..'\u0191'
| '\u0193'..'\u0194'
| '\u0196'..'\u0198'
| '\u019C'..'\u019D'
| '\u019F'..'\u01A0'
| '\u01A2'
| '\u01A4'
| '\u01A6'..'\u01A7'
| '\u01A9'
| '\u01AC'
| '\u01AE'..'\u01AF'
| '\u01B1'..'\u01B3'
| '\u01B5'
| '\u01B7'..'\u01B8'
| '\u01BC'
| '\u01C4'
| '\u01C7'
| '\u01CA'
| '\u01CD'
| '\u01CF'
| '\u01D1'
| '\u01D3'
| '\u01D5'
| '\u01D7'
| '\u01D9'
| '\u01DB'
| '\u01DE'
| '\u01E0'
| '\u01E2'
| '\u01E4'
| '\u01E6'
| '\u01E8'
| '\u01EA'
| '\u01EC'
| '\u01EE'
| '\u01F1'
| '\u01F4'
| '\u01F6'..'\u01F8'
| '\u01FA'
| '\u01FC'
| '\u01FE'
| '\u0200'
| '\u0202'
| '\u0204'
| '\u0206'
| '\u0208'
| '\u020A'
| '\u020C'
| '\u020E'
| '\u0210'
| '\u0212'
| '\u0214'
| '\u0216'
| '\u0218'
| '\u021A'
| '\u021C'
| '\u021E'
| '\u0220'
| '\u0222'
| '\u0224'
| '\u0226'
| '\u0228'
| '\u022A'
| '\u022C'
| '\u022E'
| '\u0230'
| '\u0232'
| '\u0386'
| '\u0388'..'\u038A'
| '\u038C'
| '\u038E'..'\u038F'
| '\u0391'..'\u03A1'
| '\u03A3'..'\u03AB'
| '\u03D2'..'\u03D4'
| '\u03D8'
| '\u03DA'
| '\u03DC'
| '\u03DE'
| '\u03E0'
| '\u03E2'
| '\u03E4'
| '\u03E6'
| '\u03E8'
| '\u03EA'
| '\u03EC'
| '\u03EE'
| '\u03F4'
| '\u0400'..'\u042F'
| '\u0460'
| '\u0462'
| '\u0464'
| '\u0466'
| '\u0468'
| '\u046A'
| '\u046C'
| '\u046E'
| '\u0470'
| '\u0472'
| '\u0474'
| '\u0476'
| '\u0478'
| '\u047A'
| '\u047C'
| '\u047E'
| '\u0480'
| '\u048A'
| '\u048C'
| '\u048E'
| '\u0490'
| '\u0492'
| '\u0494'
| '\u0496'
| '\u0498'
| '\u049A'
| '\u049C'
| '\u049E'
| '\u04A0'
| '\u04A2'
| '\u04A4'
| '\u04A6'
| '\u04A8'
| '\u04AA'
| '\u04AC'
| '\u04AE'
| '\u04B0'
| '\u04B2'
| '\u04B4'
| '\u04B6'
| '\u04B8'
| '\u04BA'
| '\u04BC'
| '\u04BE'
| '\u04C0'..'\u04C1'
| '\u04C3'
| '\u04C5'
| '\u04C7'
| '\u04C9'
| '\u04CB'
| '\u04CD'
| '\u04D0'
| '\u04D2'
| '\u04D4'
| '\u04D6'
| '\u04D8'
| '\u04DA'
| '\u04DC'
| '\u04DE'
| '\u04E0'
| '\u04E2'
| '\u04E4'
| '\u04E6'
| '\u04E8'
| '\u04EA'
| '\u04EC'
| '\u04EE'
| '\u04F0'
| '\u04F2'
| '\u04F4'
| '\u04F8'
| '\u0500'
| '\u0502'
| '\u0504'
| '\u0506'
| '\u0508'
| '\u050A'
| '\u050C'
| '\u050E'
| '\u0531'..'\u0556'
| '\u10A0'..'\u10C5'
| '\u1E00'
| '\u1E02'
| '\u1E04'
| '\u1E06'
| '\u1E08'
| '\u1E0A'
| '\u1E0C'
| '\u1E0E'
| '\u1E10'
| '\u1E12'
| '\u1E14'
| '\u1E16'
| '\u1E18'
| '\u1E1A'
| '\u1E1C'
| '\u1E1E'
| '\u1E20'
| '\u1E22'
| '\u1E24'
| '\u1E26'
| '\u1E28'
| '\u1E2A'
| '\u1E2C'
| '\u1E2E'
| '\u1E30'
| '\u1E32'
| '\u1E34'
| '\u1E36'
| '\u1E38'
| '\u1E3A'
| '\u1E3C'
| '\u1E3E'
| '\u1E40'
| '\u1E42'
| '\u1E44'
| '\u1E46'
| '\u1E48'
| '\u1E4A'
| '\u1E4C'
| '\u1E4E'
| '\u1E50'
| '\u1E52'
| '\u1E54'
| '\u1E56'
| '\u1E58'
| '\u1E5A'
| '\u1E5C'
| '\u1E5E'
| '\u1E60'
| '\u1E62'
| '\u1E64'
| '\u1E66'
| '\u1E68'
| '\u1E6A'
| '\u1E6C'
| '\u1E6E'
| '\u1E70'
| '\u1E72'
| '\u1E74'
| '\u1E76'
| '\u1E78'
| '\u1E7A'
| '\u1E7C'
| '\u1E7E'
| '\u1E80'
| '\u1E82'
| '\u1E84'
| '\u1E86'
| '\u1E88'
| '\u1E8A'
| '\u1E8C'
| '\u1E8E'
| '\u1E90'
| '\u1E92'
| '\u1E94'
| '\u1EA0'
| '\u1EA2'
| '\u1EA4'
| '\u1EA6'
| '\u1EA8'
| '\u1EAA'
| '\u1EAC'
| '\u1EAE'
| '\u1EB0'
| '\u1EB2'
| '\u1EB4'
| '\u1EB6'
| '\u1EB8'
| '\u1EBA'
| '\u1EBC'
| '\u1EBE'
| '\u1EC0'
| '\u1EC2'
| '\u1EC4'
| '\u1EC6'
| '\u1EC8'
| '\u1ECA'
| '\u1ECC'
| '\u1ECE'
| '\u1ED0'
| '\u1ED2'
| '\u1ED4'
| '\u1ED6'
| '\u1ED8'
| '\u1EDA'
| '\u1EDC'
| '\u1EDE'
| '\u1EE0'
| '\u1EE2'
| '\u1EE4'
| '\u1EE6'
| '\u1EE8'
| '\u1EEA'
| '\u1EEC'
| '\u1EEE'
| '\u1EF0'
| '\u1EF2'
| '\u1EF4'
| '\u1EF6'
| '\u1EF8'
| '\u1F08'..'\u1F0F'
| '\u1F18'..'\u1F1D'
| '\u1F28'..'\u1F2F'
| '\u1F38'..'\u1F3F'
| '\u1F48'..'\u1F4D'
| '\u1F59'
| '\u1F5B'
| '\u1F5D'
| '\u1F5F'
| '\u1F68'..'\u1F6F'
| '\u1FB8'..'\u1FBB'
| '\u1FC8'..'\u1FCB'
| '\u1FD8'..'\u1FDB'
| '\u1FE8'..'\u1FEC'
| '\u1FF8'..'\u1FFB'
| '\u2102'
| '\u2107'
| '\u210B'..'\u210D'
| '\u2110'..'\u2112'
| '\u2115'
| '\u2119'..'\u211D'
| '\u2124'
| '\u2126'
| '\u2128'
| '\u212A'..'\u212D'
| '\u2130'..'\u2131'
| '\u2133'
| '\u213E'..'\u213F'
| '\u2145'
| '\uFF21'..'\uFF3A'
| '\u010400'..'\u010425'
| '\u01D400'..'\u01D419'
| '\u01D434'..'\u01D44D'
| '\u01D468'..'\u01D481'
| '\u01D49C'
| '\u01D49E'..'\u01D49F'
| '\u01D4A2'
| '\u01D4A5'..'\u01D4A6'
| '\u01D4A9'..'\u01D4AC'
| '\u01D4AE'..'\u01D4B5'
| '\u01D4D0'..'\u01D4E9'
| '\u01D504'..'\u01D505'
| '\u01D507'..'\u01D50A'
| '\u01D50D'..'\u01D514'
| '\u01D516'..'\u01D51C'
| '\u01D538'..'\u01D539'
| '\u01D53B'..'\u01D53E'
| '\u01D540'..'\u01D544'
| '\u01D546'
| '\u01D54A'..'\u01D550'
| '\u01D56C'..'\u01D585'
| '\u01D5A0'..'\u01D5B9'
| '\u01D5D4'..'\u01D5ED'
| '\u01D608'..'\u01D621'
| '\u01D63C'..'\u01D655'
| '\u01D670'..'\u01D689'
| '\u01D6A8'..'\u01D6C0'
| '\u01D6E2'..'\u01D6FA'
| '\u01D71C'..'\u01D734'
| '\u01D756'..'\u01D76E'
| '\u01D790'..'\u01D7A8'
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list