[antlr-interest] Token position 0, -1

[Jim Idle]

It isn't a bug - you will just break other things. This is a known defect
because the Java code relies on overloaded methods in the runtime to
perform one call or the other. I can't do that in the C runtime and there
is no context at code generation that lets me 'know' which one is correct.

However, such rewrites are generally completely unnecessary anyway.

Jim

> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of A Z
> Sent: Tuesday, July 10, 2012 7:02 AM
> To: Mike Lischke
> Cc: antlr-interest at antlr.org
> Subject: Re: [antlr-interest] Token position 0, -1
>
> This is a bug in the C code generator. The parser code should call
> createTypeToken() but instead calls createTypeText().  You can fix it
> by modifying antlr-
> 3.4/tool/src/main/resources/org/antlr/codegen/templates/C/AST.stg
> towards the end of the file and recompiling the jar file.
>
>
> On Tue, Jul 10, 2012 at 3:55 AM, Mike Lischke <mike at lischke-
> online.de>wrote:
>
> >
> > Stefan, Jim,
> >
> > thanks for the hint about the start/stop tokens.
> >
> > >> Is there a similar construct if you have a rule instead of a
> token?
> > >> For
> > >> instance:
> > >>
> > >> target_table:
> > >>      qualified_identifier -> ^(TABLE_REF_ID qualified_identifier)
> ;
> > >
> > > I use in some case the start attribute of the subrule:
> > >
> > > target_table:
> > >       qualified_identifier ->
> > > ^(TABLE_REF_ID[$qualified_identifier.start]
> > > qualified_identifier)
> >
> > This doesn't work with my grammar (using the C target). Instead of
> > taking the location only the token text of my virtual token is set to
> > NULL. The generated C code is:
> >
> >             if ( BACKTRACKING==0 )
> >
> >             {
> >                 pANTLR3_REWRITE_RULE_SUBTREE_STREAM stream_retval;
> >
> >
> > stream_retval=antlr3RewriteRuleSubtreeStreamNewAEE(ADAPTOR,
> > RECOGNIZER, (pANTLR3_UINT8)"token retval", retval.tree != NULL ?
> > retval.tree : NULL);
> >
> >                 root_0                      =
> > (pANTLR3_BASE_TREE)(ADAPTOR->nilNode(ADAPTOR));
> >                 retval.tree    = root_0;
> >                 // 580:23: -> ^(
> > TABLE_REF_ID[$qualified_identifier.start]
> > qualified_identifier )
> >                 {
> >                     // MySQL55.g:580:26: ^(
> > TABLE_REF_ID[$qualified_identifier.start] qualified_identifier )
> >                     {
> >                         pANTLR3_BASE_TREE root_1 =
> > (pANTLR3_BASE_TREE)(ADAPTOR->nilNode(ADAPTOR));
> >                         root_1 =
> > (pANTLR3_BASE_TREE)(ADAPTOR->becomeRoot(ADAPTOR,
> >
> >                         #if 1 == 2
> >
> > (pANTLR3_BASE_TREE)ADAPTOR->createTypeTokenText(ADAPTOR,
> TABLE_REF_ID,
> > TOKTEXT((qualified_identifier695.start)))
> >                         #else
> >
> > (pANTLR3_BASE_TREE)ADAPTOR->createTypeText(ADAPTOR, TABLE_REF_ID,
> > (pANTLR3_UINT8)(qualified_identifier695.start))
> >                         #endif
> >
> >                         , root_1));
> >
> >                         ADAPTOR->addChild(ADAPTOR, root_1,
> > stream_qualified_identifier == NULL ? NULL :
> > stream_qualified_identifier->nextTree(stream_qualified_identifier));
> >
> >                         ADAPTOR->addChild(ADAPTOR, root_0, root_1);
> >                     }
> >
> >                 }
> >
> >
> >                 retval.tree = root_0; // set result root
> >
> > which shows that the start token is used only to create the token
> text
> > for root_1, which is clearly not what we want. Also, this 1 == 2 part
> > seems quite useless and is just bloating the already huge file
> (almost
> > 134K lines already).
> >
> > Mike
> > --
> > www.soft-gems.net
> >
> >
> >
> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > Unsubscribe:
> > http://www.antlr.org/mailman/options/antlr-interest/your-email-
> address
> >
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address