[antlr-interest] Possible bug in C runtime: Unable to override emit() in lexer

Jim Idle jimi at temporal-wave.com
Tue Apr 26 11:11:06 PDT 2011


No. just copy the functions to your own code set, rename and install the
pointer to the custom method after you have created the lexer.

Jim

> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of Michael Carns
> Sent: Tuesday, April 26, 2011 11:07 AM
> To: antlr-interest at antlr.org
> Subject: Re: [antlr-interest] Possible bug in C runtime: Unable to
> override emit() in lexer
>
> Jim,
>
> I'm planning on using the custom pointer to stash my data.  When I
> first looked those fields, it looked to me like they are designed to be
> set by action blocks in the grammar.  I have some common metadata that
> needs to go along with every token, and I'd rather not have to insert
> and maintain an action block for every single lexer rule.  That's what
> started me down the path of overriding emit.  It provided a common
> point where I could adjust those values.  Is there a better place to
> insert my code?
>
> Thanks,
> Mike
>
> On 4/26/2011 8:05 AM, Jim Idle wrote:
> > No, you override both of these with your own function, and your own
> > function should call your own version of emit if the built in one is
> > not what you want.
> >
> > However most people will not need to use a customized token type and
> > emit overrides and so on. There is a pointer and 3 integers in the
> > token that can be used to store any additional information that you
> need.
> >
> >
> > Jim
> >
> >> -----Original Message-----
> >> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> >> bounces at antlr.org] On Behalf Of Michael Carns
> >> Sent: Tuesday, April 26, 2011 12:42 AM
> >> To: antlr-interest at antlr.org
> >> Subject: [antlr-interest] Possible bug in C runtime: Unable to
> >> override
> >> emit() in lexer
> >>
> >> In order to use a customized token type, I believe I need to
> override
> >> emit() and emitNew() within my ANTLR3_LEXER instance.  However, my
> >> function isn't getting called.  I'm using version 3.1.3 of the C
> >> runtime.
> >>
> >> The culprit appears to be this call to emit() within nextTokenStr()
> >> in
> >> antlr3lexer.c:
> >>
> >> Line 268:
> >>                  if (lexer->rec->state->token == NULL)
> >>                  {
> >>                      // Emit the real token, which adds it in to the
> >> token stream basically
> >>                      //
> >>                      emit(lexer);
> >>                  }
> >>
> >> Shouldn't it be "lexer->emit(lexer)"?  As written, the call to
> emit()
> >> will bind to the local version of emit() within antlr3lexer.c and
> >> won't call my customized version.  I've looked at more recent
> >> versions in the source repository, and the code appears to be
> >> unchanged.  Is this a real bug, or am I going down the wrong road in
> >> my attempt to customize the generated tokens?
> >>
> >> Thanks,
> >> Mike Carns
> >>
> >>
> >> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> >> Unsubscribe:
> >> http://www.antlr.org/mailman/options/antlr-interest/your-
> >> email-address
> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > Unsubscribe:
> > http://www.antlr.org/mailman/options/antlr-interest/your-email-
> address
> >
>
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address


More information about the antlr-interest mailing list