[antlr-interest] ANTLR 3.1 gives RewriteEmptyStreamException, ANTLR3.0.1 does not ?

[David Pearce]

Hi Terence,

>> The reason is that the empty production has no rewrite.  However, in
>> ANTLR 3.0.1, this error did not happen.
>>
>> So, my question is: is this is a bug, or intentional behaviour?  I
> 
> hi!
> 
> It is proper behavior; if there is no listBody then list must indicate 
> there is an error.

Just to be clear here --- you're saying this behaviour exhibited in 
ANTLR 3.0.1 was entirely broken?  It was actually quite a useful 
behaviour.  For example, in the Java 1.5 Grammar you have:

> classDeclaration
> 	:	'class' Identifier (typeParameters)?
>         ('extends' type)? 
>         ('implements' typeList)?
>         classBody
> 	;
> 
> ...
> 
> classBody
> 	:	'{' classBodyDeclaration* '}'
> 	;
> 
> ...
> 
> classBodyDeclaration
> 	:	';'
> 	|	... [other productions]
> 	;

In my ideal AST, the tokens ';', '{' and '}' are not present.  But, to 
use the rewrite rules with this, I have to provide extra tokens like so:

 > classBody
 > 	: '{' classBodyDeclaration* '}' -> (^X classBodyDeclaration*)
 > 	;
 >
 > ...
 >
 > classBodyDeclaration
 > 	:	';' -> (^Y)
 > 	|	... [other productions]
 > 	;

This seems clumsy, since neither the X or Y tokens are actually needed 
in my final AST.  With ANTLR 3.0.1 I just left them as empty 
productions, and it worked really nicely.

One solution I guess is to refactor the grammar, by inlining the 
productions in question.  So instead of the above, after one iteration 
we get:

 > classDeclaration
 > 	:	'class' Identifier (typeParameters)?
 >         ('extends' type)?
 >         ('implements' typeList)?
 >         '{' classBodyDeclaration* '}'
 >         -> ^(CLASS ...)
 > 	;

But, I still need to inline classBodyDeclaration ... and as I do all 
this the neatness of the original grammar is being lost.

Any suggestions?  Did I miss anything?

Cheers,

Dave

> Ter

-- 
Lecturer in Computer Science,
School of Mathematics, Statistics and Computer Science,
Victoria University of Wellington,
PO Box 600,
Wellington,
New Zealand.

Office: Cotton 231
Telephone: +64 (0)4 463 5833
URL: http://www.mcs.vuw.ac.nz/~djp