[antlr-interest] Python How To Convert from Java

[Bob Adolf]

Hi Remy,

I think you'll be surprised at how similar the python and java  
versions actually are. You made things a little more difficult for  
yourself than you needed. I've put some comments in-line with your  
original python code and finished off the end.

> import antlr3
> import antlr3.tree
> from sqltestLexer import sqltestLexer
> from sqltestParser import sqltestParser
> #import sqltest ?                                    won't import
The grammar sqltest.g actually doesn't produce an sqltest file or  
class, so you just need the first two.

> import sys
> #from sqltest import sqltest
> #sys.argv[1]
> char_stream = antlr3.ANTLRStringStream("SELECT * FROM BOOKS;")
Your grammar actually doesn't parse this phrase, but I assume you just  
forgot to replace the place-holder string with something else. I used  
this instead:
char_stream = antlr3.ANTLRStringStream("CREATE TABLE foo ( bar int ) ;")
which (not knowing SQL) seems to fit your grammar at least.

It seems like you tried using an input file first. I'm assuming that  
you want something like your original java file that selects between a  
file or a string. Well, it looks very much like the java version:
if( len(sys.argv)>1 ):
   char_stream = antlr3.ANTLRFileStream(sys.argv[1])
else:
   char_stream = antlr3.ANTLRStringStream(raw_input("SQL>"))

> lexer = sqltestLexer(char_stream)
> tokens = antlr3.CommonTokenStream(lexer)
> parser = sqltestParser(tokens)
>
> r = parser.createtablestmt_return()
You'll actually want to call the function itself. The types are  
handled automatically, so you can focus on the work that's actually  
being done. The function parser.createtablestmt (which you can look at  
directly in sqltestParser.py) is ANTLR's translation of your root- 
level grammar rule. You can walk through the logic in the python  
source as it tries to match against input tokens and recursively  
invokes the other rules.
r = parser.createtablestmt()

Interestingly enough, this is where your java version stops. You just  
print out the tree element.
# System.out.println("tablename ="+ 
((Tree)r.tree).getChild(2).toString());
# System.out.println("tree="+((Tree)r.tree).toStringTree());

You can do the same thing in python!
print "tablename = " + r.tree.children[2].toString()
print "tree = " + r.tree.toStringTree()


>
> # this is the root of the AST
> root = r.tree
>
> nodes = antlr3.tree.CommonTreeNodeStream(root)
> nodes.setTokenStream(tokens)
>
> #walker = sqltest(nodes)
> #  STUCK!

As for the rest of this, it looks like you started trying to implement  
a tree walker. If you're serious about munging the AST, you'll  
probably want to do this later, but for simple things like just  
dumping the tree, you don't need to. Likewise, if you're just doing a  
simple extraction of SQL snippets or some easy translation, you can do  
it directly by inserting actions into your parser at the appropriate  
place.

If you do decide that a tree walker is what you need, then you'll need  
to create a separate grammar for that (and this is why you got stuck,  
there was nothing you could invoke because you hadn't written a tree  
parser). You should probably run through the python examples (http://www.antlr.org/download/examples-v3.tar.gz 
) or the ANTLR book if you're going to do that.

Hope this helps.


   -Bob



For clarity, here's the whole file I used (driver.py):
import antlr3
from sqltestLexer import sqltestLexer
from sqltestParser import sqltestParser
import sys

if( len(sys.argv)>1 ):
   char_stream = antlr3.ANTLRFileStream(sys.argv[1])
else:
   char_stream = antlr3.ANTLRStringStream(raw_input("SQL>"))
lexer = sqltestLexer(char_stream)
tokens = antlr3.CommonTokenStream(lexer)
parser = sqltestParser(tokens)

r = parser.createtablestmt()

# System.out.println("tablename ="+ 
((Tree)r.tree).getChild(2).toString());
# System.out.println("tree="+((Tree)r.tree).toStringTree());
print "tablename = " + r.tree.children[2].toString()
print "tree = " + r.tree.toStringTree()

And the output:
tablename = foo
tree = CREATE TABLE foo ( bar int )