[antlr-interest] Different parse tree in 3.1 Vs 3.4

Jim Idle jimi at temporal-wave.com
Tue Oct 4 13:37:25 PDT 2011


Just post it here - it will get a JIRA if it is deemed to be a bug.

Jim

> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of Ashutosh Chauhan
> Sent: Tuesday, October 04, 2011 12:13 PM
> To: antlr-interest at antlr.org
> Subject: Re: [antlr-interest] Different parse tree in 3.1 Vs 3.4
>
> I think I have a reproducible bug report for this which illustrates
> different behavior in 3.0.1 vs 3.4, where shall I report it ?
> antlr.org/jirasays contact Administrator to request an account.
>
> Thanks,
> Ashutosh
>
> On Fri, Sep 23, 2011 at 12:55, Jim Idle <jimi at temporal-wave.com> wrote:
>
> > I am saying that you should clear up this stuff everywhere first
> though.
> > Then your best bet is to generate the code with the -debug flag,
> build
> > it and run it then debug remotely with ANTLRWorks. I also sometimes
> > just use the Java debugger as the generated method names match the
> rules.
> >
> > jim
> >
> > > -----Original Message-----
> > > From: Ashutosh Chauhan [mailto:hashutosh at apache.org]
> > > Sent: Friday, September 23, 2011 12:50 PM
> > > To: Jim Idle
> > > Cc: antlr-interest at antlr.org
> > > Subject: Re: [antlr-interest] Different parse tree in 3.1 Vs 3.4
> > >
> > > Hey Jim,
> > >
> > > Thanks for taking a look. I can see what you are saying, but as far
> > > as I can see those productions will never be used in the test-case
> I
> > > am having, so those are irrelevant for the problem at hand.
> > >
> > > Thanks,
> > > Ashutosh
> > >
> > > On Fri, Sep 23, 2011 at 12:30, Jim Idle <jimi at temporal-wave.com>
> wrote:
> > > > Try the following:
> > > >
> > > > 1) You have things like:
> > > >
> > > >    : KW_RESTRICT
> > > >    -> ^(TOK_RESTRICT)
> > > >
> > > > Which are a little pointless - you could just use the existing
> > > > parser
> > > > token:
> > > >
> > > >    : KW_RESTRICT
> > > >
> > > > But writing a single token will not produce a node anyway.
> > > >
> > > > Next, you have things like this:
> > > >
> > > >    :
> > > >      keyValueProperty (COMMA keyValueProperty)* ->
> > > > ^(TOK_DBPROPLIST
> > > > keyValueProperty+)
> > > >    ;
> > > >
> > > > Which I think will need to be:
> > > >
> > > >    :
> > > >      k+=keyValueProperty (COMMA k+=keyValueProperty)* ->
> > > > ^(TOK_DBPROPLIST
> > > > $k+)
> > > >    ;
> > > >
> > > > Start with that and let's see where you get to.
> > > >
> > > > Jim
> > > >
> > > >> -----Original Message-----
> > > >> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> > > >> bounces at antlr.org] On Behalf Of Ashutosh Chauhan
> > > >> Sent: Friday, September 23, 2011 10:59 AM
> > > >> To: antlr-interest at antlr.org
> > > >> Subject: [antlr-interest] Different parse tree in 3.1 Vs 3.4
> > > >>
> > > >> Hey,
> > > >>
> > > >> I work on Apache Hive project which was using Antlr 3.0.1 till
> > > >> now and its working great. Thank you guys. Now I am trying to
> > > >> port it to
> > > >> 3.4 and I see antlr returning incorrect parse tree in 3.4 which
> > > >> used to be correct in 3.0.1. I tested with 3.1 and thats
> > > >> returning
> > > correct
> > > >> parse tree too. Grammar is defined here:
> > > >>
> > > >>
> > >
> https://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/h
> > > a
> > > >> do
> > > >> op/hive/ql/parse/Hive.g
> > > >>
> > > >> and test program is:
> > > >>
> > > >> insert overwrite table alter3 partition (pCol1='test_part:',
> > > >> pcol2='test_part:') select col1 from alter3_src ;
> > > >>
> > > >> In 3.0.1 and 3.1 I get following parse tree which is correct
> > > >> (nil(TOK_QUERY (TOK_FROM(TOK_TABREF
> > > >> (TOK_TABNAME(alter3_src))))(TOK_INSERT
> > > >>
> > >
> (TOK_DESTINATION(TOK_TAB(TOK_TABNAME(alter3))(TOK_PARTSPEC(TOK_PARTV
> > > A
> > > >> L(
> > > >>
> > >
> pCol1)('test_part:'))(TOK_PARTVAL(pcol2)('test_part:')))))(TOK_SELEC
> > > T
> > > >> (T
> > > >> OK_SELEXPR(TOK_TABLE_OR_COL(col1))))))(null))
> > > >>
> > > >> whereas in 3.4 I get following parse tree which is incomplete
> and
> > > >> thus incorrect (nil(TOK_QUERY (TOK_FROM (TOK_TABREF
> > > >> (TOK_TABNAME(alter3_src))))(TOK_INSERT
> > > >> (TOK_DESTINATION(TOK_TAB))(TOK_SELECT (TOK_SELEXPR
> > > >>       (TOK_TABLE_OR_COL(col1))))))(<EOF>))
> > > >>
> > > >> Is there something I should keep in mind while trying to upgrade
> > > >> to 3.4. I was not able to test with 3.2 or 3.3 because of a
> > > >> different problem in ant plugin of antlr because of which I was
> > > >> not able to build. It was failing following stacktrace:
> > > >>
> > > >> Exception in thread "main" java.lang.NoClassDefFoundError:
> > > >> antlr/RecognitionException
> > > >>      [java]   at org.antlr.Tool.getRootGrammar(Tool.java:551)
> > > >>      [java]   at org.antlr.Tool.process(Tool.java:422)
> > > >>      [java]   at org.antlr.Tool.main(Tool.java:91)
> > > >>      [java] Caused by: java.lang.ClassNotFoundException:
> > > >> antlr.RecognitionException
> > > >>
> > > >> I will be happy to provide any more information which will help
> > > >> figure out this issue.
> > > >>
> > > >> Thanks for your help,
> > > >> Ashutosh
> > > >>
> > > >> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > > >> Unsubscribe:
> > > >> http://www.antlr.org/mailman/options/antlr-interest/your-
> > > >> email-address
> > > >
> > > > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > > > Unsubscribe:
> > > > http://www.antlr.org/mailman/options/antlr-interest/your-email-
> > > address
> > > >
> >
> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > Unsubscribe:
> > http://www.antlr.org/mailman/options/antlr-interest/your-email-
> address
> >
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address


More information about the antlr-interest mailing list