[antlr-interest] garmmar inheritance problem

[Oleg N. Sukhodolsky]

Hi,

I've found some problems with grammar inheritance and I wonder
if they are bugs or features.  And if they are features is there are 
any workaround for them.

The problem is as follows:
I get java.g grammar from examples and create java14.g to add assert statement.
For this reason in java.g I rename statement to traditionalStatement and 
add rule:
statement : traditionalStatement;

in  java14.g I change rule for statement:
statement
	:	traditionalStatement
	|	assertStatement
	;
And add:
assertStatement
	:	"assert"^ expression ( COLON expression )? SEMI
	;

here is the first problem after compilation "assert" is not added to 
keywords hash and so lexer consider it as IDENT.
I can workaround it if I add token ASSERT to lexer and will use it in the
rule:
assertStatement
	:	ASSERT^ expression ( COLON expression )? SEMI
	;

class Java14Lexer extends JavaLexer;
...
tokens {
        ASSERT="assert";
}

Althought, this works until I add some new tokens to recognizer.  In this case
"assert" will be added to the hash with wrong id.  And this is second problem.

And the third problem is that if I want extends 14 recognizer to add 15 features
then Java15Lexer doesnt have "assert" in the keywords hash until I specify it
explicitly (but in this case it will have wrong id).

I've tested this with antlr 2.7.(2|3|4rc1).

So, the questions are:
1. Are these problems bugs or not?
2. if these are features is there any way to workaround them?

Thanks in advance, Oleg.

P.S. here are java14.g and java15.g
----- java14.g ------------
class Java14Recognizer extends JavaRecognizer;

// Options don't get inherited, copy of option block required.
options {
	k = 2;                           // two token lookahead
	exportVocab=Java14;     // Call its vocabulary "GeneratedJava14"
	codeGenMakeSwitchThreshold = 2;  // Some optimizations
	codeGenBitsetTestThreshold = 3;
	defaultErrorHandler = false;     // Don't generate parser error handlers
	buildAST = true;
}

// overrides the statement production in java.g, adds assertStatement
statement
	:	traditionalStatement
	|	assertStatement
	;

// assert statement, available since JDK 1.4
assertStatement
	:	ASSERT^ expression ( COLON expression )? SEMI
	;

class Java14Lexer extends JavaLexer;

options {
	exportVocab=Java14;    // call the vocabulary "Java14",
	testLiterals=false;    // don't automatically test for literals
	k=4;                   // four characters of lookahead
	charVocabulary='\u0003'..'\uFFFE';
	codeGenBitsetTestThreshold=20;
}

tokens {
        ASSERT="assert";
}

protected
FLOAT_SUFFIX
	:	'f'|'F'|'d'|'D'
	;
----- java14.g ------------
----- java15.g ------------
class Java15Recognizer extends Java14Recognizer;

// Options don't get inherited, copy of option block required.
options {
	k = 2;                           // two token lookahead
	exportVocab=Java15;     // Call its vocabulary "GeneratedJava15"
	codeGenMakeSwitchThreshold = 2;  // Some optimizations
	codeGenBitsetTestThreshold = 3;
	defaultErrorHandler = false;     // Don't generate parser error handlers
	buildAST = true;
}

// just some statement
statement
	:	traditionalStatement
	|	assertStatement
	;

class Java15Lexer extends Java14Lexer;

options {
	exportVocab=Java15;    // call the vocabulary "Java15",
	testLiterals=false;    // don't automatically test for literals
	k=4;                   // four characters of lookahead
	charVocabulary='\u0003'..'\uFFFE';
	codeGenBitsetTestThreshold=20;
}

protected
FLOAT_SUFFIX
	:	'f'|'F'|'d'|'D'
	;
----- java15.g ------------



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/