[antlr-interest] Accessing HIDDEN tokens in the C target.

[Robin Green]

Hello!

Longtime listener, firstime, er, caller.

I am using the "C" target and I am trying to get access to the
$channel=HIDDEN tokens from the parser, but only inside one rule. The FAQ
and advice on the mailing list that I have read talk about subclassing the
ANTLR3_COMMON_TOKEN_STREAM class and rewriting the skipOffTokenChannels()
function to send back every token in the stream.

1. It's unclear how to "subclass" the objects in the C target library,
especially how to rewrite a "member" function.

2. Even if I do so, how do I switch between token stream as I enter a rule
and restore it as I leave? I understand the examples that push the current
stream onto a stack for "including" files, but these streams are complete in
themselves, so when they swap back they only have to pick up where they left
off. I really need to switch on and off the skipping of off-channel tokens.

What's technique should I use? I am leaning towards writing my own "get()"
in the lookahead:

shader
  : ( options{greedy=false;}: ( val = ~('technique')
    {
      ANTLR3_INT32 index = $val.index + 1;
      ANTLR3_INT32 max_size = INPUT->istream->cachedSize;
      while(index < max_size) {
         pANTLR_COMMON_TOKEN token = INPUT->get(INPUT, index);
         if (token->getChannel(token) == HIDDEN) {
           // add lookahead token to the output
        }
      }
    }
    )* )
  ;

and using it inside a parser rule to directly diddle with the token stream,
but I was wondering if there is a better way that doesn't involve me
customizing the C target library (which will require re-customizing every
time I update the antlr package). Is there a way to inject my own behavior
into the C libs without recompiling it?

Thanks in advance,

- Robin Green
  Google Inc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20080828/805cbb42/attachment.html