[antlr-interest] C target - Disabling lexer output for groups of tokens

A Z asicaddress at gmail.com
Thu Apr 14 15:44:54 PDT 2011


Thanks for the response.  I couldn't find where nextToken is set but I see
it now in antlr3lexer.c

antlr3LexerNew(ANTLR3_UINT32 sizeHint, pANTLR3_RECOGNIZER_SHARED_STATE
state)
{
...
        /* Install the default nextToken() method, which may be overridden
         * by generated code, or by anything else in fact.
         */
        lexer->rec->state->tokSource->nextToken        =  nextToken;
...
}

On Thu, Apr 14, 2011 at 10:41 AM, Jim Idle <jimi at temporal-wave.com> wrote:

> Implement your own next token method. There are two functions, this one
> and nextToken - copy them and make the change then install your pointer
> before calling the lexer.
>
> Jim
>
> > -----Original Message-----
> > From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> > bounces at antlr.org] On Behalf Of A Z
> > Sent: Thursday, April 14, 2011 8:03 AM
> > To: antlr-interest at antlr.org
> > Subject: [antlr-interest] C target - Disabling lexer output for groups
> > of tokens
> >
> > Is there way to persistantly switch the lexer output on or off? Channel
> > assignments only last for one token. I know I can use skip() but then I
> > have to add the same code to every lexer rule(I have hundreds) where it
> > checks a boolean and then executes skip(). I also looked at changing
> > the default channel but I don't how that can be done as the following
> > isn't a function pointer that can be reassigned:
> >
> > ANTLR3_INLINE static pANTLR3_COMMON_TOKEN
> > nextTokenStr        (pANTLR3_TOKEN_SOURCE toksource)
> > {
> > ...
> >             lexer->rec->state->channel                        =
> > ANTLR3_TOKEN_DEFAULT_CHANNEL;
> > ...
> > }
> >
> > Is there another way of doing this?
> >
> > Thanks
> >
> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> > email-address
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>


More information about the antlr-interest mailing list