[antlr-interest] URGENT: Problems with Unicode support in ANTLR

Matthew Ford Matthew.Ford at forward.com.au
Sat May 18 16:55:51 PDT 2002


Ter,  I think the  codeGenBitsetTestThreshold=5;
is broken.

I have attached my test .g file and the lexer output.  no tokenSets were
generated.  All case statements.
matthew

the Ident rule was
IDENT_N1
 options {testLiterals=true;
     paraphrase = "N1 identifier";}
 :
('\u16EE'..'\u16F0'|'_'|'$'|'\u2160'..'\u2183'|'\u3007'..'\u3007'|'\u3021'..
'\u3029'|'\u3038'..'\u303A')

('\u16EE'..'\u16F0'|'_'|'0'..'9'|'$'|'\u2160'..'\u2183'|'\u3007'..'\u3007'|'
\u3021'..'\u3029'|'\u3038'..'\u303A')*
 ;

----- Original Message -----
From: "micheal_jor" <open.zone at virgin.net>
To: <antlr-interest at yahoogroups.com>
Sent: Sunday, May 19, 2002 3:07 AM
Subject: [antlr-interest] URGENT: Problems with Unicode support in ANTLR


> Hi,
>
> Would ANLTR gurus please respond to my queries regarding the best
> option to take with respect to this message:
>
> http://groups.yahoo.com/group/antlr-interest/message/5537
>
> It is urgent and I would appreciate any help in making the best
> choice until we can improve ANTLR's native unicode support.
>
> Cheers,
>
> Micheal
>
>
>
>
>
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
>
>

 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: Unicode.g
Type: application/octet-stream
Size: 2439 bytes
Desc: not available
Url : http://www.antlr.org/pipermail/antlr-interest/attachments/20020519/4f95a48e/Unicode.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: UnicodeLexer.java
Type: application/octet-stream
Size: 8647 bytes
Desc: not available
Url : http://www.antlr.org/pipermail/antlr-interest/attachments/20020519/4f95a48e/UnicodeLexer.obj


More information about the antlr-interest mailing list