[antlr-interest] Python How To Convert from Java

[Remy]

Bob,

Thank you very much, that was an excellent explanation in great detail.
I'm guessing the next step would be to move away from:
r.tree.children[2].toString()
to something more useful like:  tablename = r.getTableName();
Is this at all possible and where can I find an example of this?
I see a similar example here but I'm not having much luck getting it to work:
http://codemate.wordpress.com/2009/02/06/writing-a-sql-parser/

Remy.

On Fri, Feb 26, 2010 at 10:04 AM, Bob Adolf <rdadolf at gmail.com> wrote:
> Hi Remy,
>
> I think you'll be surprised at how similar the python and java versions
> actually are. You made things a little more difficult for yourself than you
> needed. I've put some comments in-line with your original python code and
> finished off the end.
>
>> import antlr3
>> import antlr3.tree
>> from sqltestLexer import sqltestLexer
>> from sqltestParser import sqltestParser
>> #import sqltest ?                                    won't import
>
> The grammar sqltest.g actually doesn't produce an sqltest file or class, so
> you just need the first two.
>
>> import sys
>> #from sqltest import sqltest
>> #sys.argv[1]
>> char_stream = antlr3.ANTLRStringStream("SELECT * FROM BOOKS;")
>
> Your grammar actually doesn't parse this phrase, but I assume you just
> forgot to replace the place-holder string with something else. I used this
> instead:
> char_stream = antlr3.ANTLRStringStream("CREATE TABLE foo ( bar int ) ;")
> which (not knowing SQL) seems to fit your grammar at least.
>
> It seems like you tried using an input file first. I'm assuming that you
> want something like your original java file that selects between a file or a
> string. Well, it looks very much like the java version:
> if( len(sys.argv)>1 ):
>  char_stream = antlr3.ANTLRFileStream(sys.argv[1])
> else:
>  char_stream = antlr3.ANTLRStringStream(raw_input("SQL>"))
>
>> lexer = sqltestLexer(char_stream)
>> tokens = antlr3.CommonTokenStream(lexer)
>> parser = sqltestParser(tokens)
>>
>> r = parser.createtablestmt_return()
>
> You'll actually want to call the function itself. The types are handled
> automatically, so you can focus on the work that's actually being done. The
> function parser.createtablestmt (which you can look at directly in
> sqltestParser.py) is ANTLR's translation of your root-level grammar rule.
> You can walk through the logic in the python source as it tries to match
> against input tokens and recursively invokes the other rules.
> r = parser.createtablestmt()
>
> Interestingly enough, this is where your java version stops. You just print
> out the tree element.
> # System.out.println("tablename ="+((Tree)r.tree).getChild(2).toString());
> # System.out.println("tree="+((Tree)r.tree).toStringTree());
>
> You can do the same thing in python!
> print "tablename = " + r.tree.children[2].toString()
> print "tree = " + r.tree.toStringTree()
>
>
>>
>> # this is the root of the AST
>> root = r.tree
>>
>> nodes = antlr3.tree.CommonTreeNodeStream(root)
>> nodes.setTokenStream(tokens)
>>
>> #walker = sqltest(nodes)
>> #  STUCK!
>
> As for the rest of this, it looks like you started trying to implement a
> tree walker. If you're serious about munging the AST, you'll probably want
> to do this later, but for simple things like just dumping the tree, you
> don't need to. Likewise, if you're just doing a simple extraction of SQL
> snippets or some easy translation, you can do it directly by inserting
> actions into your parser at the appropriate place.
>
> If you do decide that a tree walker is what you need, then you'll need to
> create a separate grammar for that (and this is why you got stuck, there was
> nothing you could invoke because you hadn't written a tree parser). You
> should probably run through the python examples
> (http://www.antlr.org/download/examples-v3.tar.gz) or the ANTLR book if
> you're going to do that.
>
> Hope this helps.
>
>
>  -Bob
>
>
>
> For clarity, here's the whole file I used (driver.py):
> import antlr3
> from sqltestLexer import sqltestLexer
> from sqltestParser import sqltestParser
> import sys
>
> if( len(sys.argv)>1 ):
>  char_stream = antlr3.ANTLRFileStream(sys.argv[1])
> else:
>  char_stream = antlr3.ANTLRStringStream(raw_input("SQL>"))
> lexer = sqltestLexer(char_stream)
> tokens = antlr3.CommonTokenStream(lexer)
> parser = sqltestParser(tokens)
>
> r = parser.createtablestmt()
>
> # System.out.println("tablename ="+((Tree)r.tree).getChild(2).toString());
> # System.out.println("tree="+((Tree)r.tree).toStringTree());
> print "tablename = " + r.tree.children[2].toString()
> print "tree = " + r.tree.toStringTree()
>
> And the output:
> tablename = foo
> tree = CREATE TABLE foo ( bar int )
>
>