[antlr-interest] RewriteEmptyStreamException when changing from quoted string to token?

[Mark Wright]

On Tue, 15 Jan 2008 08:30:58 +0000
David Holroyd <dave at badgers-in-foil.co.uk> wrote:

> On Mon, Jan 14, 2008 at 05:40:10PM -0600, Jon Schewe wrote:
> > David Holroyd wrote:
> > > On Mon, Jan 14, 2008 at 02:26:26PM -0600, Jon Schewe wrote:
> > >   
> > >> I just got time to get back to this project, been stuck on a
> > >> different one.  I tried your suggestion of gathering everything
> > >> up in a variable and that didn't help.  Still the same error. 
> > >>
> > >> /**
> > >> * @param negate if true, negate all expressions by multiplying
> > >> by -1 */   
> > >> multExpr[boolean negate]
> > >>    :   a+=atom (PRODUCT a+=atom)* -> {negate}? ^(PRODUCT
> > >> ^(NUMBER DOUBLE["-1"]) $a) ->           ^(PRODUCT $a)
> > >>    ;
> > >>
> > >>    [junit]     Caused an ERROR
> > >>    [junit] token PRODUCT
> > >>    [junit] org.antlr.runtime.tree.RewriteEmptyStreamException:
> > >> token PRODUCT 
> > >
> > > You can't fix just by writing PRODUCT* on the r.h.s., as a tree
> > > with zero-or-more root nodes doesn't make sense.
> > >
> > > Also, do you really always want to create a tree with PRODUCT as
> > > the root here, even if PRODUCT is not in the input?  I would have
> > > thought that if no PRODUCT is present, you'd just want 'atom'?
> > >
> > >   
> > Yes, I really want to always have PRODUCT, at least that's how my
> > tree walker is setup right now.  It makes for a more regular tree.
> > Is the problem that antlr can't figure out which PRODUCT token to
> > use?
> 
> Ah, sorry.
> 
> Yes, my understanding is that ANTLR sees that there are PRODUCT
> entries on the left and right of the rewrite, and therefor tracks the
> PRODUCTs seen while parsing in order to stick those actual entries in
> to the rewritten tree.

Hello Dave,

That seems like you might be on the right line of thinking.
I don't actually know, but I'll continue to sort of speculate
anyway...
 
> I suspect RewriteEmptyStreamException is the result when the rewrite
> tries to insert one of the captured PRODUCTs, but the parse didn't
> capture one (because there was an 'atom' with no '*', I guess).

I am sort of wondering if maybe ANTLR might be expecting
to see all the PRODUCT tokens that were matched in the
rule appear in the AST rewrite rule, and maybe its confused
when there is only 1 in the AST rewrite rule?

That guess could easilly be tested by changing the PRODUCT
in the rewrite rule to an imaginary token.

Crazy idea: I don't suppose ANTLR would like .start attribute here
(from p. 130 figure 6.2 of the ANTLR book)

multExpr[boolean negate]
   :   a+=atom (PRODUCT a+=atom)* -> {negate}? ^(PRODUCT.start ^(NUMBER DOUBLE["-1"]) $a)  
                                  ->           ^(PRODUCT.start $a)  
   ;

> Here's another idea that I didn't test :)
>
>   multExpr[boolean negate]
>     :   a+=atom (a+=multHelp)* -> {negate}? ^(PRODUCT ^(NUMBER
> DOUBLE["-1"]) $a) ->           ^(PRODUCT $a)
>     ;
> 
>   multHelp
>     :   PRODUCT atom -> atom

I wonder if that might be different as it seems like the
a+=multHelp might be trying to collect a sequence
of PRODCUT atom tokens?

Anyway I'm only guessing, so feel free to ignore me.

Thanks, Mark