[antlr-interest] charVocabulary having no effect

André Næss andre.naess at gmail.com
Mon Dec 13 06:16:02 PST 2004



I'm struggling a bit with charVocabulary. After getting lot's of
strange "unexpected character" errors I figured that this was a rather
important option. I therefore added

    charVocabulary = '\3'..'\377';

To my Lexer options.

But I'm still getting unexpected char errors. I have a fairly simple
grammar with a non-greedy rule to match the contents of a specific
portion. When the lexer encounters the char '=' in this portion it
stops saying "unexpected character". If I then add this:

POINTLESS : '=' ;

The error goes away, but then it stops on some other char. This
continues until I've added all the chars not listed in some rule in
the lexer. So to be sure it seems I will have to explicitly list *all*
the ASCII characters.

Grepping through the generated code I could not find a single
reference to "charVocabulary" or "vocabulary". Is this option broken?

I'm using Antlr 2.7.4 on Linux (Mandrake 10.0) with Java 1.4.2.

The lexer definition from the grammar file:

class QuerySchemaLexer extends Lexer;
options {
    charVocabulary = '\3'..'\377';
 	caseSensitiveLiterals = false;
}

RPAREN : ')';
LPAREN : '(';
COLON  : ':';
SEMI   : ';';
COMMA  : ',';

ID
options {
  testLiterals = true;
  paraphrase = "an identifer";
}
	:	('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'_'|'0'..'9')*
	;


WS  :   (   ' '
        |   '\t'
        |   '\r' '\n' { newline(); }
        |   '\n'      { newline(); }
        )
        {$setType(Token.SKIP);} //ignore this token
    ;





 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
    antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 





More information about the antlr-interest mailing list