[antlr-interest] Fundamental tree parsing question
Ted Villalba
ted.villalba at gmail.com
Tue Jul 10 18:31:58 PDT 2007
Hi,
I thought I would reuse this thread since it seems a similar issue, or
misunderstanding on my part...
When I add a rewrite specification to my grammar rule it suddenly fails
with a .RewriteEmptyStreamException.
The full grammar is below, but here is what I think it boils down to - I'm
not sure how to represent zero or more instances in the rewrite of an
operator when it is the root node .
For example, the following grammar is parsed successfully and ANTLRWorks
builds a flat tree.
value : terms ((WS)+ operator^ (WS)+ value)*
But when I try to add the following rewrite specification, I get the
exception:
value : terms ((WS)+ operator (WS)+ value)* -> ^(operator ^(TERMS terms)
^(VALUE value)*)
((((((
I created that rewrite because I thought I had read where if the operator (
or other) node is null, it won't err out, it will simply not add a node.
I may be misguided here. It's how I interpreted the code snippet at the
bottom of this wiki on AST:
http://www.antlr.org/wiki/pages/viewpage.action?pageId=754
it says:
/** Add t as a child to this node. If t is null, do nothing. If t
* is nil, add all children of t to this' children.
*/
void addChild(Tree t);
))))))
At the end of parsing:
PY=(hey hey what AND can I do)
I get:
Exception in thread "main"
org.antlr.runtime.tree.RewriteEmptyStreamException: rule operator
at org.antlr.runtime.tree.RewriteRuleElementStream._next(Unknown Source)
at org.antlr.runtime.tree.RewriteRuleSubtreeStream.nextNode(Unknown
Source)
at WQLParser.value(WQLParser.java:1195)
at WQLParser.value(WQLParser.java:1162)
at WQLParser.field(WQLParser.java:724)
at WQLParser.query(WQLParser.java:253)
at WQLParser.start(WQLParser.java:127)
at __Test__.main(__Test__.java:14)
I've tried a bunch of combinations of that rewrite spec, but I can't get it
to work with any of them.
Once again thanks for the help.
Here is the full grammar:
grammar TQL;
options{
output=AST;
ASTLabelType=CommonTree;
}
tokens{ TAG; VALUE; TERMS;} //imaginary token types
@header{
import java.util.HashMap;
}
@members {
HashMap fieldMap = new HashMap();
}
start : ( query {System.out.println("AST:\n"+$query.tree.toStringTree());}
)+
;
query : field ((WS!)+ BOOL_OP^ (WS!)+ query)*
| LPAREN! query RPAREN! ((WS!)* BOOL_OP^ (WS!)* query)*
;
field : tag '=' LPAREN (WS)* value (WS)* RPAREN -> ^('=' tag value)
| tag '=' terms+ -> ^('=' tag terms)
| qid
;
value : terms ((WS)+ operator (WS)+ value)* -> ^(operator ^(TERMS terms)
^(VALUE value)*)
| LPAREN! value RPAREN! ((WS)+ operator^ (WS!)+ value)*
;
tag : WCHAR -> ^(TAG WCHAR)
;
terms : WCHAR (WS WCHAR)* -> WCHAR+
| QUOTE WCHAR (WS! WCHAR)* QUOTE
;
qid : '#'!DIGIT
;
operator: BOOL_OP|WOK_OP;
BOOL_OP : 'AND'|'and'|'OR'|'or'|'NOT'|'not';
WOK_OP : 'SAME'|'same'|'NEAR'('/'DIGIT+)*|'near'('/'DIGIT+)*;
DIGIT : ('0'..'9');
WS : (' '|'\t'|'\r'|'\n')+ ;
LPAREN : '(' ;
RPAREN : ')' ;
QUOTE : '"';
WCHAR : ~('='|'('| ')'|'"'|' '|'\t'|'\n'|'\r'|'#')+;
-Ted
On 7/10/07, Ted Villalba <ted.villalba at gmail.com> wrote:
>
> I was on 1.0.11b. Having upgraded to the latest version it works.
> Thanks for your time Benjamin.
> Very much appreciated!
>
> Ted
>
> On 7/10/07, Benjamin Niemann <pink at odahoda.de> wrote:
> >
> > Ted Villalba wrote:
> >
> > > Here is the grammar attached below, with your most recent suggestion.
> > I
> > > can walk it without issue, with input like :
> > > query=(city AND mountain dwellers)
> > >
> > > until I try to add the use of the imaginary tokens, for example, in
> > the
> > > 'field' rule below:
> > > The grammar:
> > > [...]
> >
> > Mmm... I can't reproduce the problem. I can step through the input and
> > get
> > the output:
> >
> > AST:
> > (= (TAG (TAG query)) (VALUE (AND (TERMS city) (TERMS mountain
> > dwellers))))
> >
> > Probably not what you finally want (redundant TAG and some WS not
> > omitted),
> > but at least it does not crash...
> >
> > I used ANTLRWorks 1.0.2 and "Java(TM) 2 Runtime Environment, Standard
> > Edition (build 1.5.0_12-b04)" on Debian GNU/Linux. What's your setup?
> >
> > --
> > Benjamin Niemann
> > Email: pink at odahoda dot de
> > WWW: http://pink.odahoda.de/
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20070710/41cff760/attachment.html
More information about the antlr-interest
mailing list