[antlr-interest] Fundamental tree parsing question

[Ted Villalba]

Hi,

I thought I would reuse this thread since it seems a similar issue, or
misunderstanding on my part...
 When I add a rewrite specification to my grammar rule it suddenly fails
with a .RewriteEmptyStreamException.
 The full grammar is below, but here is what I think it boils down to - I'm
not sure how to represent zero or more instances in the rewrite of an
operator when it is the root node .
For example, the following grammar is parsed successfully and ANTLRWorks
builds a flat tree.
value   : terms ((WS)+ operator^ (WS)+ value)*

But when I try to add the following rewrite specification, I get the
exception:
value : terms ((WS)+ operator (WS)+ value)* -> ^(operator ^(TERMS terms)
^(VALUE value)*)

((((((
I created that rewrite because I thought I had read where if the operator (
or other) node is null, it won't err out, it will simply not add a node.
  I may be misguided here. It's how I interpreted the code snippet at the
bottom of this wiki on AST:
http://www.antlr.org/wiki/pages/viewpage.action?pageId=754
it says:

/** Add t as a child to this node.  If t is null, do nothing.  If t
 *  is nil, add all children of t to this' children.
 */

void addChild(Tree t);

))))))


At the end of parsing:
PY=(hey hey what AND can I do)

I get:
Exception in thread "main"
org.antlr.runtime.tree.RewriteEmptyStreamException: rule operator
    at org.antlr.runtime.tree.RewriteRuleElementStream._next(Unknown Source)
    at org.antlr.runtime.tree.RewriteRuleSubtreeStream.nextNode(Unknown
Source)
    at WQLParser.value(WQLParser.java:1195)
    at WQLParser.value(WQLParser.java:1162)
    at WQLParser.field(WQLParser.java:724)
    at WQLParser.query(WQLParser.java:253)
    at WQLParser.start(WQLParser.java:127)
    at __Test__.main(__Test__.java:14)

I've tried a bunch of combinations of that rewrite spec, but I can't get it
to work with any of them.
Once again thanks for the help.

Here is the full grammar:

grammar TQL;

options{
    output=AST;
    ASTLabelType=CommonTree;
}

tokens{ TAG; VALUE; TERMS;} //imaginary token types

@header{
import java.util.HashMap;
}

@members {

HashMap fieldMap = new HashMap();

}

start   : ( query  {System.out.println("AST:\n"+$query.tree.toStringTree());}
)+
         ;


query   : field ((WS!)+ BOOL_OP^ (WS!)+ query)*
           | LPAREN! query RPAREN! ((WS!)* BOOL_OP^ (WS!)* query)*
           ;

field     : tag '=' LPAREN (WS)* value (WS)* RPAREN -> ^('=' tag value)
           | tag '=' terms+ -> ^('=' tag terms)
           | qid
           ;

value   : terms ((WS)+ operator (WS)+ value)* -> ^(operator ^(TERMS terms)
^(VALUE value)*)
           | LPAREN! value RPAREN! ((WS)+ operator^ (WS!)+ value)*
           ;

tag    : WCHAR -> ^(TAG WCHAR)
         ;

terms    : WCHAR (WS WCHAR)*  -> WCHAR+
            | QUOTE WCHAR (WS! WCHAR)* QUOTE
            ;


qid     : '#'!DIGIT
        ;

operator: BOOL_OP|WOK_OP;
BOOL_OP    : 'AND'|'and'|'OR'|'or'|'NOT'|'not';
WOK_OP    : 'SAME'|'same'|'NEAR'('/'DIGIT+)*|'near'('/'DIGIT+)*;
DIGIT   : ('0'..'9');
WS      : (' '|'\t'|'\r'|'\n')+ ;
LPAREN    : '(' ;
RPAREN    : ')' ;
QUOTE    : '"';
WCHAR   : ~('='|'('| ')'|'"'|' '|'\t'|'\n'|'\r'|'#')+;



-Ted



On 7/10/07, Ted Villalba <ted.villalba at gmail.com> wrote:
>
> I was on 1.0.11b. Having upgraded to the latest version it works.
> Thanks for your time Benjamin.
> Very much appreciated!
>
> Ted
>
> On 7/10/07, Benjamin Niemann <pink at odahoda.de> wrote:
> >
> > Ted Villalba wrote:
> >
> > > Here is the grammar attached below, with your most recent suggestion.
> > I
> > > can walk it without issue, with input like :
> > > query=(city AND mountain dwellers)
> > >
> > > until I try to add the use of the imaginary tokens, for example, in
> > the
> > > 'field' rule below:
> > > The grammar:
> > > [...]
> >
> > Mmm... I can't reproduce the problem. I can step through the input and
> > get
> > the output:
> >
> > AST:
> > (= (TAG (TAG query)) (VALUE (AND (TERMS city)     (TERMS mountain
> > dwellers))))
> >
> > Probably not what you finally want (redundant TAG and some WS not
> > omitted),
> > but at least it does not crash...
> >
> > I used ANTLRWorks 1.0.2 and "Java(TM) 2 Runtime Environment, Standard
> > Edition (build 1.5.0_12-b04)" on Debian GNU/Linux. What's your setup?
> >
> > --
> > Benjamin Niemann
> > Email: pink at odahoda dot de
> > WWW: http://pink.odahoda.de/
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20070710/41cff760/attachment.html