[antlr-interest] Different parse tree in 3.1 Vs 3.4

[Jim Idle]

I am saying that you should clear up this stuff everywhere first though.
Then your best bet is to generate the code with the -debug flag, build it
and run it then debug remotely with ANTLRWorks. I also sometimes just use
the Java debugger as the generated method names match the rules.

jim

> -----Original Message-----
> From: Ashutosh Chauhan [mailto:hashutosh at apache.org]
> Sent: Friday, September 23, 2011 12:50 PM
> To: Jim Idle
> Cc: antlr-interest at antlr.org
> Subject: Re: [antlr-interest] Different parse tree in 3.1 Vs 3.4
>
> Hey Jim,
>
> Thanks for taking a look. I can see what you are saying, but as far as
> I can see those productions will never be used in the test-case I am
> having, so those are irrelevant for the problem at hand.
>
> Thanks,
> Ashutosh
>
> On Fri, Sep 23, 2011 at 12:30, Jim Idle <jimi at temporal-wave.com> wrote:
> > Try the following:
> >
> > 1) You have things like:
> >
> >    : KW_RESTRICT
> >    -> ^(TOK_RESTRICT)
> >
> > Which are a little pointless - you could just use the existing parser
> > token:
> >
> >    : KW_RESTRICT
> >
> > But writing a single token will not produce a node anyway.
> >
> > Next, you have things like this:
> >
> >    :
> >      keyValueProperty (COMMA keyValueProperty)* -> ^(TOK_DBPROPLIST
> > keyValueProperty+)
> >    ;
> >
> > Which I think will need to be:
> >
> >    :
> >      k+=keyValueProperty (COMMA k+=keyValueProperty)* ->
> > ^(TOK_DBPROPLIST
> > $k+)
> >    ;
> >
> > Start with that and let's see where you get to.
> >
> > Jim
> >
> >> -----Original Message-----
> >> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> >> bounces at antlr.org] On Behalf Of Ashutosh Chauhan
> >> Sent: Friday, September 23, 2011 10:59 AM
> >> To: antlr-interest at antlr.org
> >> Subject: [antlr-interest] Different parse tree in 3.1 Vs 3.4
> >>
> >> Hey,
> >>
> >> I work on Apache Hive project which was using Antlr 3.0.1 till now
> >> and its working great. Thank you guys. Now I am trying to port it to
> >> 3.4 and I see antlr returning incorrect parse tree in 3.4 which used
> >> to be correct in 3.0.1. I tested with 3.1 and thats returning
> correct
> >> parse tree too. Grammar is defined here:
> >>
> >>
> https://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/ha
> >> do
> >> op/hive/ql/parse/Hive.g
> >>
> >> and test program is:
> >>
> >> insert overwrite table alter3 partition (pCol1='test_part:',
> >> pcol2='test_part:') select col1 from alter3_src ;
> >>
> >> In 3.0.1 and 3.1 I get following parse tree which is correct
> >> (nil(TOK_QUERY (TOK_FROM(TOK_TABREF
> >> (TOK_TABNAME(alter3_src))))(TOK_INSERT
> >>
> (TOK_DESTINATION(TOK_TAB(TOK_TABNAME(alter3))(TOK_PARTSPEC(TOK_PARTVA
> >> L(
> >>
> pCol1)('test_part:'))(TOK_PARTVAL(pcol2)('test_part:')))))(TOK_SELECT
> >> (T
> >> OK_SELEXPR(TOK_TABLE_OR_COL(col1))))))(null))
> >>
> >> whereas in 3.4 I get following parse tree which is incomplete and
> >> thus incorrect (nil(TOK_QUERY (TOK_FROM (TOK_TABREF
> >> (TOK_TABNAME(alter3_src))))(TOK_INSERT
> >> (TOK_DESTINATION(TOK_TAB))(TOK_SELECT (TOK_SELEXPR
> >>       (TOK_TABLE_OR_COL(col1))))))(<EOF>))
> >>
> >> Is there something I should keep in mind while trying to upgrade to
> >> 3.4. I was not able to test with 3.2 or 3.3 because of a different
> >> problem in ant plugin of antlr because of which I was not able to
> >> build. It was failing following stacktrace:
> >>
> >> Exception in thread "main" java.lang.NoClassDefFoundError:
> >> antlr/RecognitionException
> >>      [java]   at org.antlr.Tool.getRootGrammar(Tool.java:551)
> >>      [java]   at org.antlr.Tool.process(Tool.java:422)
> >>      [java]   at org.antlr.Tool.main(Tool.java:91)
> >>      [java] Caused by: java.lang.ClassNotFoundException:
> >> antlr.RecognitionException
> >>
> >> I will be happy to provide any more information which will help
> >> figure out this issue.
> >>
> >> Thanks for your help,
> >> Ashutosh
> >>
> >> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> >> Unsubscribe:
> >> http://www.antlr.org/mailman/options/antlr-interest/your-
> >> email-address
> >
> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > Unsubscribe:
> > http://www.antlr.org/mailman/options/antlr-interest/your-email-
> address
> >