[antlr-interest] Problem with vocabs, inheritance and literals

[Michael Studman]

Hi.

 

I'm working on adapting the Java 1.5 grammar I submitted to antlr.org
for the Checkstyle project. Checkstyle has java 1.3 and 1.4 grammars
where 1.4 lexer/recognizer derives from 1.3 lexer/recognizer; this is
how I've adapted the 1.5 grammar - by deriving from 1.4 and overriding
rules where necessary.

 

The problem I'm having is that the "enum" literal referenced in the 1.5
grammar is not added to the literal map in lexer's generated constructor
and consequently is mistaken by the lexer for an IDENT with value
"enum".

 

I'm pretty sure it's got to do with my flawed understanding of
vocabularies and inheritance so here's a description of the grammars and
vocabs and how they relate to each other:

 

In java.g there is:

class GeneratedJavaRecognizer extends Parser;

 

options {

            exportVocab=GeneratedJava;

.

}

.

 

class GeneratedJavaLexer extends Lexer;

 

options {

            exportVocab=GeneratedJava;

.

}

.

 

In java14.g there is:

 

class GeneratedJava14Recognizer extends GeneratedJavaRecognizer;

 

options {

            exportVocab=GeneratedJava14;

.

}

.

 

class GeneratedJava14Lexer extends GeneratedJavaLexer;

 

options {

            exportVocab=GeneratedJava14;

.

}

.

 

And in java15.g there is:

 

class GeneratedJava15Recognizer extends GeneratedJavar14Recognizer;

 

options {

            exportVocab=GeneratedJava15;

.

}

.

 

class GeneratedJava15Lexer extends GeneratedJava14Lexer;

 

options {

            exportVocab=GeneratedJava15;

.

}

.

 

 

This causes the GeneratedJava15TokenTypes.txt and the
GeneratedJava15TokenTypes.java both to have a LITERAL_enum entry but, as
I said the GeneratedJava15Lexer constructor doesn't register this as a
literal.

 

If I change the GeneratedJava15Lexer's definition to:

 

class GeneratedJava15Lexer extends GeneratedJava14Lexer;

 

options {

            importVocab=GeneratedJava15;

.

}

.

 

the problem goes away (the lexer registers the literal) but this seems
counter intuitive to my understanding of how vocabs work within files
and through inheritance. Incidentally, this change also causes a
GenerateJava15LexerTokenTypes.txt/.java file to be created - presumably
because the export vocab of the lexer defaults to its own name.

 

Any advice is appreciated.

 

Regards,

Michael.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20040804/a3d7b3c8/attachment.html