[antlr-interest] Accessing HIDDEN tokens in the C target.

Jim Idle jimi at temporal-wave.com
Thu Aug 28 17:07:12 PDT 2008


On Thu, 2008-08-28 at 17:01 -0700, Robin Green wrote:
> 
> 
> 
> On Thu, Aug 28, 2008 at 4:38 PM, Jim Idle <jimi at temporal-wave.com>
> wrote:
> 
>         On Thu, 2008-08-28 at 16:33 -0700, Jim Idle wrote:
>         
>         > When you want to do this in one rule, just use get it
>         > directly starting from the current index. Say you want to
>         > look back from a particular token in a rule to see if there
>         > was a comment on channel 2 (please note that I have not
>         > compiled this, just typed it in from memory). Basically
>         > though you can do anything you can do in Java (more in
>         > fact), just by looking a ta Java example and realizing that
>         > the C methods are all the same names (just about) but
>         > instead of  x.y(z), you use x->y(x, z).
>         > 
>         > rule
>         > : f=FUNCTION x y z
>         >     {
>         >         int sIndex;
>         >        pANTLR3_COMMON_TOKEN tok;
>         > 
>         >        sIndex = $f->getTokenIndex($f) - 1;   // Index for
>         > first token
>         >   
>         >       // Now look back up looking for tokens
>         >      tok = INPUT->get(INPUT, sIndex);
>         
>         
>         
>         
>         Actually, this should be:
>         
>         
>         tok = (pANTLR3_COMMON_TOKEN)(INPUT->get(INPUT, sIndex)->super);
>         
>         Jim
> 
> 
> 
> Are you sure? "super" is not a member of ANTLR3_COMMON_TOKEN_struct.
> Which object are you "super"ing here, the Token or the TokenStream?


Yes but get returns a pointer to a TOKEN, which has a super pointer to
the common token (off the top of my head), if it is returning a
pANTLR3_COMMON_TOKEN already (it might be) then you don't need the
super.

> 
> 
> The code for "get()" in antlr3tokenstream.c takes the result of the
> INPUT macro, follows the super pointer to reach a
> pANTLR3_COMMON_TOKEN_STREAM, then directly accesses the pANTLR3_VECTOR
> "tokens" to get the next symbol. What are you suggesting that is
> different from that?

The macros stop you having to learn too much about the internals, which
can change between releases like Hogwart's staircases.

> 
> 
> Mixing these macros with structure fields is making my head spin.


A lot easier than ctx->rec->ts->cts->get

;-) Just takes a few goes and you will be fluent in it.

Jim
> 
> 
> - Robin Green
>   Google, Inc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20080828/b3baf49a/attachment.html 


More information about the antlr-interest mailing list