[antlr-interest] updating text dump when writing tree grammar ast transformers

[Leo Meyerovich]

Maybe that was too verbose. How should I write the following? Special 
emphasis on having the $text attribute satisfying the equality for 
potential downstream tree grammars.

tree grammar ES3Simplifier ;

options
{
    ASTLabelType = CommonTree ;
    tokenVocab = ES3;
    output=AST;
}
...
expr
   : ^( ASSIGN a=expr b=expr )  {$text = $b.text + " = " + $a.text;} ->
        ^( ASSIGN {$b.tree} {$a.tree})

I'm starting to think ANTLR 3.1 cannot handle this. Is there at least a 
recommended way to do these AST transforms in Java such that printing is 
'for free'? Also, for trivial rewrites (like the above swap) into the 
same source language, StringTemplates will often suffice, but I need 
them to generate ASTs - is there a way to do this without needed to 
relex and parse the full input file after?

- Leo

Leo Meyerovich wrote:
> Hiya,
>
> I'm using Patrick Hulsmeijer's JavaScript grammar & tree walker. I 
> running a 3 stage pipeline: a grammar to parse, a tree grammar walker 
> to transform (output=ast), and then a tree grammar walker to pretty 
> print. While I verified that the AST structure is being transformed, I 
> can't reuse the existing top $text attribute to easily print - it 
> dumps the untransformed AST! More specifically:
>
> //1. parser: unmodified es3
>
> //2. transformer: modified es3walker with the following change to flip 
> assignments:
> expr:
>   ...
>    | ^( ASSIGN a=expr b=expr )  -> ^(ASSIGN $b $a)        ...
>
> //3. printer: modified to print at two locations, top, and @ point of 
> transformation
>   program : (statement {System.out.println($statement.text);})* ;
> ...
> expr:
>   ...
>    | ^( ASSIGN a=expr b=expr )  {System.out.println("//=, " + $a.text 
> + ", " + $b.text);}
>  ...
>
> While the print out associated with the assignment rule prints out as 
> expected (reversed assignment), the whole AST dump in the program rule 
> shows the unmodified source. I tried playing a bit with token settings 
> as described near the end of chapter 7 (albeit not really for the 
> then-nonexistent tree rewrites) such as writing "-> ^(ASSIGN $b[$b, 
> $b.text] ...)", but did not bump into anything that compiled.
>
> Will I need to write an explicit pretty printer action for every rule, 
> or is there a way to do tree transforms that will propagate associated 
> text?
>
> As a side note, has anybody had success with rewrite=true for tree 
> grammar transformers? My AST gets butchered when I add a transform to 
> the assignment arm of the expression rule (perhaps I just need to add 
> explicit identity transforms to the rest of the arms of the expression 
> rule?). I suspect inplace transforms would be best for my task so it'd 
> be nice :)
>
> This is all on build 2008-04-09.18.
>
> Thanks!
>
> - Leo
>
> (ps: I'll post my transforms and a short explanation when/if I get 
> this working, which hopefully will aid the coming release of this very 
> useful feature)