[antlr-interest] Tree Rewriting problem - RewriteEmptyStreamException at runtime

[Jon Schewe]

I don't know if one is more correct than the other.  I've just found
that it seems to work better to have everything defined as tokens. 

As far as the exception goes, I'm not sure, other than not doing the
rewrite in the tree grammar.  Is there any reason you can't do this in
the parser?

Grzegorz Cieslewski wrote:
> My mistake,  I have been experimenting with more complex grammar and
> accidentally pasted that error.
> The error I am getting in the grammar attached it
>
> Exception in thread "main"
> org.antlr.runtime.tree.RewriteEmptyStreamException: rule type
>         at org.antlr.runtime.tree.RewriteRuleElementStream._next(RewriteRuleElementStream.java:158)
>         at org.antlr.runtime.tree.RewriteRuleElementStream.nextTree(RewriteRuleElementStream.java:145)
>         at TP.a(TP.java:125)
>         at Main.main(Main.java:37)
>
> I am working on a c parser (and more), I used the ANSI C grammar by
> Terence that is on the website as a start point.
> In that grammar he has a rule called type specifiers which looks like:
>
> type_specifier
> 	: 'void'
> 	| 'char'
> 	| 'short'
> 	| 'int'
> 	| 'long'
> 	| 'float'
> 	| 'double'
> 	| 'signed'
> 	| 'unsigned'
> 	| struct_or_union_specifier
> 	| enum_specifier
> 	| type_id
> 	;
>
> He does not defines them as tokens either.  I guess I am just little
> confused about which way is the correct way?
>
>
>
> On Feb 7, 2008 7:10 AM, Jon Schewe <jpschewe at mtu.net> wrote:
>   
>> I typically don't do rewrites in my tree grammar, but in my parser.  So
>> that might be a problem.  The other thing that I've noticed helps with
>> these errors is to make sure to define tokens for all elements.  So
>> 'int' and 'char' should be defined in your lexer as tokens and then
>> reference the tokens in your parser and tree grammar.
>>
>> It also seems that the exception doesn't match your grammar.  The errors
>> simpleCrw.declaration and simpleCrw.file suggest to me that you have a
>> grammar named simpleCrw with rules declaration and file and that's where
>> the error is.  However those rules don't exist in the file you sent.
>>
>>
>> Grzegorz Cieslewski wrote:
>>     
>>> I am using the 3.1b1 version of antlr so I could do some tree
>>> rewriting.  I found a good set of examples in the
>>> TestTreeGrammarRewriteAST.java file and decided to start
>>> experimenting.  Unfortunately I have run in to a problem with my
>>> program throwing an exception.
>>> My Grammar is:
>>> grammar T;
>>> options {output=AST;}
>>> tokens
>>> {
>>>       EXPR;
>>> }
>>> a : type ID INT -> ^(EXPR type ID INT);
>>> type  :       'int'
>>>       |       'char';
>>> ID : 'a'..'z'+ ;
>>> INT : '0'..'9'+;
>>> WS : (' '|'\n') {$channel=HIDDEN;} ;
>>>
>>> and the tree grammar:
>>>
>>> tree grammar TP;
>>> options {
>>>       output=AST;
>>>       ASTLabelType=CommonTree;
>>>       tokenVocab=T;}
>>>
>>> a     : ^(EXPR type ID INT) -> ^(EXPR type INT ID)
>>>       ;
>>>
>>> type  :       'int'
>>>       |       'char';
>>>
>>> f
>>> Everything compiles but I get following error during runtime (with
>>> input "int abc 123"):
>>>
>>> Exception in thread "main"
>>> org.antlr.runtime.tree.RewriteEmptyStreamException: rule type
>>>         at org.antlr.runtime.tree.RewriteRuleElementStream._next(RewriteRuleElementStream.java:158)
>>>         at org.antlr.runtime.tree.RewriteRuleElementStream.nextTree(RewriteRuleElementStream.java:145)
>>>         at simpleCrw.declaration(simpleCrw.java:182)
>>>         at simpleCrw.file(simpleCrw.java:92)
>>>         at Main2.main(Main2.java:41)
>>>
>>> I have noticed that if I change the rule type in the tree grammar as
>>> follows the problem does not occur:
>>>
>>> tree grammar TP;
>>> options {
>>>       output=AST;
>>>       ASTLabelType=CommonTree;
>>>       tokenVocab=T;}
>>>
>>> a     : ^(EXPR type ID INT) -> ^(EXPR type INT ID)
>>>       ;
>>>
>>> type  :       'int';
>>>
>>> I have tracked this a little further.  When I remove the rewrite rule
>>> from the tree grammar :
>>>
>>> tree grammar TP;
>>> options {
>>>       output=AST;
>>>       ASTLabelType=CommonTree;
>>>       tokenVocab=T;}
>>>
>>> a     : ^(EXPR type ID INT)
>>>       ;
>>>
>>> type  :       'int'
>>>       |       'char';
>>>
>>> The output tree I get after the LexerParser is
>>> EXPR
>>> |-int
>>> |-abc
>>> |-123
>>>
>>> but after the tree grammar I get the tree missing the int node
>>> EXPR
>>> |-abc
>>> |-123
>>>
>>> Does any one have any idea as to what is going on?
>>>
>>> P.S. I attached my whole setup
>>>
>>>
>>>
>>>       
>> --
>> Jon Schewe | http://mtu.net/~jpschewe
>> If you see an attachment named signature.asc, this is my digital
>> signature.
>> See http://www.gnupg.org for more information.
>>
>> For I am convinced that neither death nor life, neither angels
>> nor demons, neither the present nor the future, nor any
>> powers, neither height nor depth, nor anything else in all
>> creation, will be able to separate us from the love of God that
>> is in Christ Jesus our Lord. - Romans 8:38-39
>>
>>
>>     
>
>
>
>   

-- 
Jon Schewe | http://mtu.net/~jpschewe
If you see an attachment named signature.asc, this is my digital
signature.
See http://www.gnupg.org for more information.

For I am convinced that neither death nor life, neither angels 
nor demons, neither the present nor the future, nor any 
powers, neither height nor depth, nor anything else in all 
creation, will be able to separate us from the love of God that 
is in Christ Jesus our Lord. - Romans 8:38-39