[antlr-interest] Different parse tree in 3.1 Vs 3.4

Ashutosh Chauhan hashutosh at apache.org
Fri Sep 23 10:58:58 PDT 2011


Hey,

I work on Apache Hive project which was using Antlr 3.0.1 till now and
its working great. Thank you guys. Now I am trying to port it to 3.4
and I see antlr returning incorrect parse tree in 3.4 which used to be
correct in 3.0.1. I tested with 3.1 and thats returning correct parse
tree too. Grammar is defined here:

https://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g

and test program is:

insert overwrite table alter3 partition (pCol1='test_part:',
pcol2='test_part:') select col1 from alter3_src ;

In 3.0.1 and 3.1 I get following parse tree which is correct
(nil(TOK_QUERY (TOK_FROM(TOK_TABREF
(TOK_TABNAME(alter3_src))))(TOK_INSERT
(TOK_DESTINATION(TOK_TAB(TOK_TABNAME(alter3))(TOK_PARTSPEC(TOK_PARTVAL(pCol1)('test_part:'))(TOK_PARTVAL(pcol2)('test_part:')))))(TOK_SELECT(TOK_SELEXPR(TOK_TABLE_OR_COL(col1))))))(null))

whereas in 3.4 I get following parse tree which is incomplete and thus incorrect
(nil(TOK_QUERY (TOK_FROM (TOK_TABREF
(TOK_TABNAME(alter3_src))))(TOK_INSERT
(TOK_DESTINATION(TOK_TAB))(TOK_SELECT (TOK_SELEXPR
	(TOK_TABLE_OR_COL(col1))))))(<EOF>))

Is there something I should keep in mind while trying to upgrade to
3.4. I was not able to test with 3.2 or 3.3 because of a different
problem in ant plugin of antlr because of which I was not able to
build. It was failing following stacktrace:

Exception in thread "main" java.lang.NoClassDefFoundError:
antlr/RecognitionException
     [java] 	at org.antlr.Tool.getRootGrammar(Tool.java:551)
     [java] 	at org.antlr.Tool.process(Tool.java:422)
     [java] 	at org.antlr.Tool.main(Tool.java:91)
     [java] Caused by: java.lang.ClassNotFoundException:
antlr.RecognitionException

I will be happy to provide any more information which will help figure
out this issue.

Thanks for your help,
Ashutosh


More information about the antlr-interest mailing list