[antlr-interest] Tree Rewriting problem - RewriteEmptyStreamException at runtime
Grzegorz Cieslewski
cieslewski at hcs.ufl.edu
Thu Feb 7 08:10:28 PST 2008
I believe that it is possible to do the rewrites I need in the parser
but I wanted a solution that is little more robust. (I do not know
the full scope of my project yet)
As to the error I explicitly defined tokens for 'int' and 'char' but
the error still occurs.
grammar T;
options {output=AST;}
tokens
{
EXPR;
}
a : type ID INT -> ^(EXPR type ID INT);
type : TYPE_INT
| TYPE_CHAR;
TYPE_INT
: 'int';
TYPE_CHAR
: 'char';
ID : 'a'..'z'+ ;
INT : '0'..'9'+;
WS : (' '|'\n') {$channel=HIDDEN;} ;
tree grammar TP;
options {
output=AST;
ASTLabelType=CommonTree;
tokenVocab=T;}
a : ^(EXPR type ID INT) -> ^(EXPR type INT ID)
;
type : TYPE_INT
| TYPE_CHAR;
On Feb 7, 2008 10:50 AM, Jon Schewe <jpschewe at mtu.net> wrote:
> I don't know if one is more correct than the other. I've just found
> that it seems to work better to have everything defined as tokens.
>
> As far as the exception goes, I'm not sure, other than not doing the
> rewrite in the tree grammar. Is there any reason you can't do this in
> the parser?
>
>
> Grzegorz Cieslewski wrote:
> > My mistake, I have been experimenting with more complex grammar and
> > accidentally pasted that error.
> > The error I am getting in the grammar attached it
> >
> > Exception in thread "main"
> > org.antlr.runtime.tree.RewriteEmptyStreamException: rule type
> > at org.antlr.runtime.tree.RewriteRuleElementStream._next(RewriteRuleElementStream.java:158)
> > at org.antlr.runtime.tree.RewriteRuleElementStream.nextTree(RewriteRuleElementStream.java:145)
> > at TP.a(TP.java:125)
> > at Main.main(Main.java:37)
> >
> > I am working on a c parser (and more), I used the ANSI C grammar by
> > Terence that is on the website as a start point.
> > In that grammar he has a rule called type specifiers which looks like:
> >
> > type_specifier
> > : 'void'
> > | 'char'
> > | 'short'
> > | 'int'
> > | 'long'
> > | 'float'
> > | 'double'
> > | 'signed'
> > | 'unsigned'
> > | struct_or_union_specifier
> > | enum_specifier
> > | type_id
> > ;
> >
> > He does not defines them as tokens either. I guess I am just little
> > confused about which way is the correct way?
> >
> >
> >
> > On Feb 7, 2008 7:10 AM, Jon Schewe <jpschewe at mtu.net> wrote:
> >
> >> I typically don't do rewrites in my tree grammar, but in my parser. So
> >> that might be a problem. The other thing that I've noticed helps with
> >> these errors is to make sure to define tokens for all elements. So
> >> 'int' and 'char' should be defined in your lexer as tokens and then
> >> reference the tokens in your parser and tree grammar.
> >>
> >> It also seems that the exception doesn't match your grammar. The errors
> >> simpleCrw.declaration and simpleCrw.file suggest to me that you have a
> >> grammar named simpleCrw with rules declaration and file and that's where
> >> the error is. However those rules don't exist in the file you sent.
> >>
> >>
> >> Grzegorz Cieslewski wrote:
> >>
> >>> I am using the 3.1b1 version of antlr so I could do some tree
> >>> rewriting. I found a good set of examples in the
> >>> TestTreeGrammarRewriteAST.java file and decided to start
> >>> experimenting. Unfortunately I have run in to a problem with my
> >>> program throwing an exception.
> >>> My Grammar is:
> >>> grammar T;
> >>> options {output=AST;}
> >>> tokens
> >>> {
> >>> EXPR;
> >>> }
> >>> a : type ID INT -> ^(EXPR type ID INT);
> >>> type : 'int'
> >>> | 'char';
> >>> ID : 'a'..'z'+ ;
> >>> INT : '0'..'9'+;
> >>> WS : (' '|'\n') {$channel=HIDDEN;} ;
> >>>
> >>> and the tree grammar:
> >>>
> >>> tree grammar TP;
> >>> options {
> >>> output=AST;
> >>> ASTLabelType=CommonTree;
> >>> tokenVocab=T;}
> >>>
> >>> a : ^(EXPR type ID INT) -> ^(EXPR type INT ID)
> >>> ;
> >>>
> >>> type : 'int'
> >>> | 'char';
> >>>
> >>> f
> >>> Everything compiles but I get following error during runtime (with
> >>> input "int abc 123"):
> >>>
> >>> Exception in thread "main"
> >>> org.antlr.runtime.tree.RewriteEmptyStreamException: rule type
> >>> at org.antlr.runtime.tree.RewriteRuleElementStream._next(RewriteRuleElementStream.java:158)
> >>> at org.antlr.runtime.tree.RewriteRuleElementStream.nextTree(RewriteRuleElementStream.java:145)
> >>> at simpleCrw.declaration(simpleCrw.java:182)
> >>> at simpleCrw.file(simpleCrw.java:92)
> >>> at Main2.main(Main2.java:41)
> >>>
> >>> I have noticed that if I change the rule type in the tree grammar as
> >>> follows the problem does not occur:
> >>>
> >>> tree grammar TP;
> >>> options {
> >>> output=AST;
> >>> ASTLabelType=CommonTree;
> >>> tokenVocab=T;}
> >>>
> >>> a : ^(EXPR type ID INT) -> ^(EXPR type INT ID)
> >>> ;
> >>>
> >>> type : 'int';
> >>>
> >>> I have tracked this a little further. When I remove the rewrite rule
> >>> from the tree grammar :
> >>>
> >>> tree grammar TP;
> >>> options {
> >>> output=AST;
> >>> ASTLabelType=CommonTree;
> >>> tokenVocab=T;}
> >>>
> >>> a : ^(EXPR type ID INT)
> >>> ;
> >>>
> >>> type : 'int'
> >>> | 'char';
> >>>
> >>> The output tree I get after the LexerParser is
> >>> EXPR
> >>> |-int
> >>> |-abc
> >>> |-123
> >>>
> >>> but after the tree grammar I get the tree missing the int node
> >>> EXPR
> >>> |-abc
> >>> |-123
> >>>
> >>> Does any one have any idea as to what is going on?
> >>>
> >>> P.S. I attached my whole setup
> >>>
> >>>
> >>>
> >>>
> >> --
> >> Jon Schewe | http://mtu.net/~jpschewe
> >> If you see an attachment named signature.asc, this is my digital
> >> signature.
> >> See http://www.gnupg.org for more information.
> >>
> >> For I am convinced that neither death nor life, neither angels
> >> nor demons, neither the present nor the future, nor any
> >> powers, neither height nor depth, nor anything else in all
> >> creation, will be able to separate us from the love of God that
> >> is in Christ Jesus our Lord. - Romans 8:38-39
> >>
> >>
> >>
> >
> >
> >
> >
>
> --
> Jon Schewe | http://mtu.net/~jpschewe
> If you see an attachment named signature.asc, this is my digital
> signature.
> See http://www.gnupg.org for more information.
>
> For I am convinced that neither death nor life, neither angels
> nor demons, neither the present nor the future, nor any
> powers, neither height nor depth, nor anything else in all
> creation, will be able to separate us from the love of God that
> is in Christ Jesus our Lord. - Romans 8:38-39
>
>
--
=====================================================
Grzegorz Cieslewski
Research Assistant
High-performance Computing & Simulation (HCS) Research Laboratory
University of Florida, Dept. of Electrical and Computer Engineering
330 Benton Hall, Gainesville, FL, 32611-6200
Phone: (352) 392-9041
Email: cieslewski at hcs.ufl.edu
Web: www.hcs.ufl.edu
=====================================================
More information about the antlr-interest
mailing list