[antlr-interest] Time for another question about Unicode support

[David Ewing]

Ric,

That's pretty much where I thought things were. I had read your other
message already, but had overlooked your reference to ICU. I actually found
out about ICU by looking at the jikes sources - I also need to get it to
support different encodings on Mac OS X.

Personally, I'd say if you're going to depend on an external library for
Unicode, ICU is the way to go. Unfortunately, I don't know the internals of
the antlr library well enough to take this on alone. At least not
considering my time constraints. Writing a lexer that handles Unicode is
probably less than a week's worth of work, since my task of parsing Java is
so narrow in scope (compared to generic Unicode support in antlr). Adding
generic support is probably an order of magnitude more work. If there had
been enough work done to give me a head start, I might have been able to
take it on. But that isn't the case. Oh well.

Thanks,
Dave

on 10/24/01 3:24 AM, Ric Klaren at klaren at cs.utwente.nl wrote:
> On Tue, Oct 23, 2001 at 11:29:17PM -0600, David Ewing wrote:
>> the C++ side. So, has using ICU been considered for ANTLR?
> 
> I've looked at it and at a few others. (but ICU looked quite nice maybe
> nicest), then again I don't know if I want a dependency on some external
> library... (without support in antlr's (c++) codegen to switch between
> support libraries etc.)
> 
> So far I've ditched any attempts at unicode for C++. (I have no personal
> interest in it, no interest from the project I'm working on (so my boss
> won't sponsor it) and the subject is way to hairy (and uninteresting) to
> spent my free time on)
> 
>> I may be able to help out in this effort, though for me that would mean
>> starting work on it soon. My guess is that my time pressures will mean
>> writing a custom lexer to deal with Unicode. Something that would return IDs
>> with UTF-8 strings. But I'd rather not do it that way. I'd rather help out
>> adding the support "the right way".
> 
> If you are willing to really look into this than I can only cheer you on =)
> and help
> 
>> Anyhow, any info or recommendations would be greatly appreciated.
> 
> See this post/thread for some thoughts I spewed out on this in the past:
> 
> http://groups.yahoo.com/group/antlr-interest/message/3973


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/