[antlr-interest] Different parse tree in 3.1 Vs 3.4

Ashutosh Chauhan hashutosh at apache.org
Tue Oct 4 12:13:04 PDT 2011


I think I have a reproducible bug report for this which illustrates
different behavior in 3.0.1 vs 3.4, where shall I report it ?
antlr.org/jirasays contact Administrator to request an account.

Thanks,
Ashutosh

On Fri, Sep 23, 2011 at 12:55, Jim Idle <jimi at temporal-wave.com> wrote:

> I am saying that you should clear up this stuff everywhere first though.
> Then your best bet is to generate the code with the -debug flag, build it
> and run it then debug remotely with ANTLRWorks. I also sometimes just use
> the Java debugger as the generated method names match the rules.
>
> jim
>
> > -----Original Message-----
> > From: Ashutosh Chauhan [mailto:hashutosh at apache.org]
> > Sent: Friday, September 23, 2011 12:50 PM
> > To: Jim Idle
> > Cc: antlr-interest at antlr.org
> > Subject: Re: [antlr-interest] Different parse tree in 3.1 Vs 3.4
> >
> > Hey Jim,
> >
> > Thanks for taking a look. I can see what you are saying, but as far as
> > I can see those productions will never be used in the test-case I am
> > having, so those are irrelevant for the problem at hand.
> >
> > Thanks,
> > Ashutosh
> >
> > On Fri, Sep 23, 2011 at 12:30, Jim Idle <jimi at temporal-wave.com> wrote:
> > > Try the following:
> > >
> > > 1) You have things like:
> > >
> > >    : KW_RESTRICT
> > >    -> ^(TOK_RESTRICT)
> > >
> > > Which are a little pointless - you could just use the existing parser
> > > token:
> > >
> > >    : KW_RESTRICT
> > >
> > > But writing a single token will not produce a node anyway.
> > >
> > > Next, you have things like this:
> > >
> > >    :
> > >      keyValueProperty (COMMA keyValueProperty)* -> ^(TOK_DBPROPLIST
> > > keyValueProperty+)
> > >    ;
> > >
> > > Which I think will need to be:
> > >
> > >    :
> > >      k+=keyValueProperty (COMMA k+=keyValueProperty)* ->
> > > ^(TOK_DBPROPLIST
> > > $k+)
> > >    ;
> > >
> > > Start with that and let's see where you get to.
> > >
> > > Jim
> > >
> > >> -----Original Message-----
> > >> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> > >> bounces at antlr.org] On Behalf Of Ashutosh Chauhan
> > >> Sent: Friday, September 23, 2011 10:59 AM
> > >> To: antlr-interest at antlr.org
> > >> Subject: [antlr-interest] Different parse tree in 3.1 Vs 3.4
> > >>
> > >> Hey,
> > >>
> > >> I work on Apache Hive project which was using Antlr 3.0.1 till now
> > >> and its working great. Thank you guys. Now I am trying to port it to
> > >> 3.4 and I see antlr returning incorrect parse tree in 3.4 which used
> > >> to be correct in 3.0.1. I tested with 3.1 and thats returning
> > correct
> > >> parse tree too. Grammar is defined here:
> > >>
> > >>
> > https://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/ha
> > >> do
> > >> op/hive/ql/parse/Hive.g
> > >>
> > >> and test program is:
> > >>
> > >> insert overwrite table alter3 partition (pCol1='test_part:',
> > >> pcol2='test_part:') select col1 from alter3_src ;
> > >>
> > >> In 3.0.1 and 3.1 I get following parse tree which is correct
> > >> (nil(TOK_QUERY (TOK_FROM(TOK_TABREF
> > >> (TOK_TABNAME(alter3_src))))(TOK_INSERT
> > >>
> > (TOK_DESTINATION(TOK_TAB(TOK_TABNAME(alter3))(TOK_PARTSPEC(TOK_PARTVA
> > >> L(
> > >>
> > pCol1)('test_part:'))(TOK_PARTVAL(pcol2)('test_part:')))))(TOK_SELECT
> > >> (T
> > >> OK_SELEXPR(TOK_TABLE_OR_COL(col1))))))(null))
> > >>
> > >> whereas in 3.4 I get following parse tree which is incomplete and
> > >> thus incorrect (nil(TOK_QUERY (TOK_FROM (TOK_TABREF
> > >> (TOK_TABNAME(alter3_src))))(TOK_INSERT
> > >> (TOK_DESTINATION(TOK_TAB))(TOK_SELECT (TOK_SELEXPR
> > >>       (TOK_TABLE_OR_COL(col1))))))(<EOF>))
> > >>
> > >> Is there something I should keep in mind while trying to upgrade to
> > >> 3.4. I was not able to test with 3.2 or 3.3 because of a different
> > >> problem in ant plugin of antlr because of which I was not able to
> > >> build. It was failing following stacktrace:
> > >>
> > >> Exception in thread "main" java.lang.NoClassDefFoundError:
> > >> antlr/RecognitionException
> > >>      [java]   at org.antlr.Tool.getRootGrammar(Tool.java:551)
> > >>      [java]   at org.antlr.Tool.process(Tool.java:422)
> > >>      [java]   at org.antlr.Tool.main(Tool.java:91)
> > >>      [java] Caused by: java.lang.ClassNotFoundException:
> > >> antlr.RecognitionException
> > >>
> > >> I will be happy to provide any more information which will help
> > >> figure out this issue.
> > >>
> > >> Thanks for your help,
> > >> Ashutosh
> > >>
> > >> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > >> Unsubscribe:
> > >> http://www.antlr.org/mailman/options/antlr-interest/your-
> > >> email-address
> > >
> > > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > > Unsubscribe:
> > > http://www.antlr.org/mailman/options/antlr-interest/your-email-
> > address
> > >
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>


More information about the antlr-interest mailing list