[antlr-interest] ANTLR 3.1 gives RewriteEmptyStreamException, ANTLR3.0.1 does not ?

[David Pearce]

Hi all,

I have a grammar which works with ANTLR 3.0.1, but no longer works with 
  ANTLR 3.1.  In particular, it seems to boil down to a strange 
difference between the way the tree rewrite rules work.

Here's the simplest grammar I could construct which illustrates the problem:

> grammar Test;
> 
> options {
>  output=AST;
>  k=2; 
>  backtrack=true; 
>  memoize=true;
> }
> 
> tokens {
>  LIST;
> }
> 
> list : '(' listBody ')' -> ^(LIST listBody)
>      ;
> 
> listBody : list -> list
>          | 
> 	 ;
> 
> WS  :  (' '|'\r'|'\t'|'\u000C'|'\n') {$channel=HIDDEN;}
>     ;

Now, if I use some boiler plate code to create a lexer/parser (see end 
of email), and then fire the following input into it "(())", I get the 
following error:

> org.antlr.runtime.tree.RewriteEmptyStreamException: rule listBody

The reason is that the empty production has no rewrite.  However, in 
ANTLR 3.0.1, this error did not happen.

So, my question is: is this is a bug, or intentional behaviour?  I 
basically want to know before I invest time fixing up my grammar!  And, 
while the above is easy enough to fix, the situations where this arises 
in my real grammar are much more awkward ...

So, any help would be really appreciated!

Cheers,

Dave

-- 
Lecturer in Computer Science,
School of Mathematics, Statistics and Computer Science,
Victoria University of Wellington,
PO Box 600,
Wellington,
New Zealand.

Office: Cotton 231
Telephone: +64 (0)4 463 5833
URL: http://www.mcs.vuw.ac.nz/~djp


>         TestLexer lex = new TestLexer(new ANTLRFileStream(args[0]));
>        	CommonTokenStream tokens = new CommonTokenStream(lex);
> 	
>         TestParser parser = new TestParser(tokens);
> 
> 	// now, attempt to parse the input file
> 	
>         try {
>             parser.list();
> 	    System.out.println("input parsed.");
>         } catch (RuntimeException e)  {
> 	    // if we get here, then there was a problem parsing the
> 	    // input file.
> 	    System.out.println("input NOT parsed.");
> 	    System.out.println(e);
>         }