[antlr-interest] TreeRewriter with Python

[Butani, Harish]

Hi,

Reading 'Language Implementation Patterns', enjoying it very much.
I wanted to try the rewriting e.g.s using Python.
I don't see a 3.2 version available, so wrote a quick and dirty rewriter
to work with version 3.1.3, it's attached. 
Tested with the Reduce grammar from the book, it works.

- had to write a new Visitor to do the child replacement
- the Rewriter is not the same as java because 'failed' is not tracked
in the state object for 3.1.3
- I just catch Backtrackingfailed and ignore it.

The changed Reduce grammar from the walking chapter is below. (the
backtracking option doesn't work in Python generation; the generated
code doesn't compile. Because applyOnce sets backtracking, I assume it
is ok not set the backtracking option in the grammar). Simple Test below
it

Anybody tried Tree rewriting with Python? Does anybody see issues with
the attached Rewriter? I would like to use it in my Project, but wanted
to get feedback on this before diving deeper; especially if there is a
major issue with this approach. 

Thanks for your help.

Regards,
Harish

 <<treerewriter.py>> 

The Grammar:
--------------------
tree grammar Reduce;

options {
    language=Python;
    output=AST;
    ASTLabelType=CommonTree;
    tokenVocab=VecMath;
    superClass=TreeRewriter;
    //backtrack=true;          // allow backtracking if it's needed
}

@header {
from antlr3.treerewriter import TreeRewriter
}

bottomup:  
  xPlusx |
  multBy2 |
  combineShifts 
;

xPlusx: ^('+' i=INT j=INT {int($i.text) == int($j.text)}?) ->
^(MULT["*"] INT["2"] $j);

multBy2
    :   ^('*' x=INT {int($x.text) == 2}? y=.) -> ^(SHIFT["<<"] $y
INT["1"])
    |   ^('*' a=. b=INT {int($b.text)==2}?) -> ^(SHIFT["<<"] $a
INT["1"])
    ;

combineShifts 
    :  ^(SHIFT ^(SHIFT e=. n=INT) m=INT)
       -> ^(SHIFT["<<"] $e INT[str(int($n.text) + int($m.text))])
  ;

Test Code:
---------------
from antlr3 import *
from antlr3.tree import *
from VecMathLexer import VecMathLexer
from VecMathParser import VecMathParser
from Reduce import Reduce

cStream = StringStream("print (3 + 3) * 2 * 2")
lexer = VecMathLexer(cStream)
tStream = CommonTokenStream(lexer)
parser = VecMathParser(tStream)
t = parser.prog().tree
print "Original tree: "+ t.toStringTree()
nodes = CommonTreeNodeStream(t)
red = Reduce(nodes);
t = red.downup(t, False)
print "Simplified tree: " + t.toStringTree()
-------------- next part --------------
A non-text attachment was scrubbed...
Name: treerewriter.py
Type: application/octet-stream
Size: 3354 bytes
Desc: treerewriter.py
Url : http://www.antlr.org/pipermail/antlr-interest/attachments/20100506/46520c3c/attachment.obj