[antlr-interest] Python target v3 unicode problems

[Viðar Svansson]

Hi,

I am trying to scan a UTF-8 file using the python target but with no luck.
I usually get this error:

UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position
0: ordinal not in range(128)

I have tried to cast the strings to unicode everywhere but I nothing
seems to work. I have also tried many different declarations of the
unicode tokens but nothing seems to work there either.

I found reference to something like this:

class L extends Lexer;
options {
	charVocabulary = '\3'..'\377' | '\u1000'..'\u1fff';
}

But I think this is v2 syntax, correct?
Does anyone have a working unicode lexer in python?

Viðar