[antlr-interest] C++ and Unicode
Ric Klaren
klaren at cs.utwente.nl
Mon Aug 16 02:09:32 PDT 2004
Hi,
On Sat, Aug 14, 2004 at 02:26:52PM +0200, Martin Probst wrote:
> I'm about to start writing a parser which has to support Unicode. I want
> to use ANTLR for this task and the output has to be Unicode. I was quite
> surprised to see that currently Unicode with C++ and ANTLR is not really
> supported. The only thing I found about it is the patch or special
> distribution (?) by Ric Klaren.
>
> Can somebody point me to information about the current status of C++
> Unicode support in ANTLR? Is the Unicode version on this page:
> <http://wwwhome.cs.utwente.nl/~klaren/antlr/right.html> in a usable
> state or more some kind of development?
The patch on my page is a hack there's a better approach detailed a while
ago by Mark Lentczner.
See this thread:
http://groups.yahoo.com/group/antlr-interest/messages/11772
I also got another 'hack' that makes the C++ part read UTF8 and store it in
the backend in std::string but UTF8 encoded. The framework for that can be
adapted quite easily to deal with other input encodings and output
encodings.
I'm waiting for some patches Mark promised me a while back. After that I'll
release a new snapshot with 2.7.4 bugfixes, the C++ port of the
TokenStreamRewriteEngine, UnicodeCharBuffer and UnicodeCharScanner. Also a
new reference counter will be used for tokens (for starters). Character
literals are limited to \ufffe though due to the ANTLR 2 analysis engine.
Cheers,
Ric
--
-----+++++*****************************************************+++++++++-------
---- Ric Klaren ----- j.klaren at utwente.nl ----- +31 53 4893755 ----
-----+++++*****************************************************+++++++++-------
Words fly like arrows
as if we knew what was right and wrong. --- Chuang Tsu
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/antlr-interest/
<*> To unsubscribe from this group, send an email to:
antlr-interest-unsubscribe at yahoogroups.com
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list