[antlr-interest] org.antlr.runtime.tree.RewriteEmptyStreamException

[David Maier]

Hello Jim,


I am still not understanding why I am expecting wrong things. As shown in my previous example that (1) and (2) are both equivalent to (3) regarding the top level rule 'a' (please see below for further details about (1), (2) and (3)). So this proofs especially that (1) is equivalent to (2). So why is it wrong to expect that the generated parsers of (1) and (2) are also equivalent? In my case with equivalent parsers I mean that the output AST of parser (1) and (2) is the same for every input. This isn't the case for my grammar because my rule in style of (1) does produce an empty AST (indeed caused by this raised exception ) and a rule in style of (2) produces the following one: http://community.ingres.com/wiki/Image:SimpleSelectMySQLAST.jpg . 

There was also an argument that it is related to the rewriting of rules. This is also mysterious for me because as far as I am understanding the rewriting of a rule simply means to substitute the right side of a rule with the new rewritten rule. So let's assume that the rewriting causes that (1) and (2) are looking as below. So what I guess is that the error occurs because a Subtree (in the AST) created by rule 'b' is expected which is finally empty. This subtree is not expected if I 'tell' ANTLR that it is optional by using the (?) operator. So the issue here is that even the fact that b can end up in the empty word means that it is 'optional' and so ANTLR should be able to handle this. So one way to solve this would be to catch the 'org.antlr.runtime.tree.RewriteEmptyStreamException' and so not to include the subtree which is generated by 'b'.

I indeed could understand if you tell me that my problem is caused by an ANTLR limitation. So then there are the two options. Option one is that it is documented somewhere as a limitation, and I simply did not search deep enough to find it. Option two is that it is not documented and so it is an ANTLR related issue, right?

Here the example rules to which I am refering in the text above:

(1) 
 a : b;
 b : c*;


(2)
 a: b?;
 b: c*;


(3)
 a: /*empty*/ | c+


Regards, David

--
David Maier
Senior Software Engineer

Ingres Germany GmbH
Weimarer Straße 1a
D-98693 Ilmenau

PHONE:  +49.3677.6785-59
FAX:    +49.3677.6785-23
MAIL:   david.maier at ingres.com

This transmission is confidential and intended solely for the use of the recipient named above. It may contain confidential, proprietary, or legally privileged information. If you are not the intended recipient, you are hereby notified that any unauthorized review, use, disclosure or distribution is strictly prohibited. If you have received this transmission in error, please contact the sender by reply e-mail and delete the original transmission and all copies from your system.



-----Ursprüngliche Nachricht-----
Von: antlr-interest-bounces at antlr.org im Auftrag von Jim Idle
Gesendet: Mi 07.07.2010 19:42
Cc: antlr-interest at antlr.org
Betreff: Re: [antlr-interest]org.antlr.runtime.tree.RewriteEmptyStreamException
 


> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of David Maier
> Sent: Wednesday, July 07, 2010 10:18 AM
> To: Andrew Haley
> Cc: antlr-interest at antlr.org
> Subject: Re: [antlr-interest]
> org.antlr.runtime.tree.RewriteEmptyStreamException
> 
> Hi Andrew,
> 
> yes it worked by using the (?) operator on one level above. However, it
> seems that the problem does occur dependent on the input. So there may
> be other rules in my grammar those are affected. So this is the reason
> why I would appreciate that it works as expected.

It works as it should, but your expectations are not correct I am afraid. Generally you are falling over on style:


X : a* b -> ^(NODE a* b) ;

Rather than

x : a b  -> ^(NODE a* b) ;
a: x* ;

Then what you think happens does happen. It also removes extra method calls (generally).

Jim






List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address