[antlr-interest] AST Rewrite, Creating imaginary tokens from real ones. C target problem. Help!!

[Vamsi Juvvi]

Hi all:

  I have more or less finished a generator for a custom IDL format used at
my company. Pretty much everything is working except when I started to
initialize imaginary tokens for AST creation from real tokens.

I am trying to accomplish the following. All C/C++ style comments in the IDL
file being parsed are placed on a hidden channel and are ignored nicely by
the parser. This works very well. I however, need to retain some comments
(those listed just before an IDL interface declaration for eg.) and so am
thinking of doing it thusly.

In the parser grammar, when creating the AST root for my interface. I use an
imaginary token which will somehow encode the line number of the first token
that matched the parser rule. My understanding is that this can be doine
useing the rule attribute $start. Then, in my treeParser, I can lookup the
line number from the imaginary token (INTERFACE_BLOCK as listed below) and
start scanning the token stream from the parser in reverse to look for
comments. I am not sure if there is a better way ?

The following is a parser rule which I am using nd which has been
frustrating me for a while now.


interface
@init { aidl::PushParserRuleOnStack("in interface declaration block");}
@after{ aidl::PopParserRuleFromStack();}
    : interface_decl
        OPEN_BRACE
            interface_body_line*
        CLOSE_BRACE SEMI_COLON
    -> ^(INTERFACE_BLOCK[$start] interface_decl interface_body_line*)
    ;


The C code generated (wihch fails to compile) for the node creation of
INTERFACE_BLOCK is as follows (there is a #if 1 == 2, #else, #endif block
which inserts lines 2 and 3 of the following 4 line block which I removed
for readability)

root_1 = (pANTLR3_BASE_TREE)(ADAPTOR->becomeRoot(ADAPTOR,

(pANTLR3_BASE_TREE)ADAPTOR->createTypeText(ADAPTOR, INTERFACE_BLOCK,
(pANTLR3_UINT8)((pANTLR3_COMMON_TOKEN)retval).start)
                        , root_1));


Which while not very easy to read is basically creating a brand new token
from the text of the "start"token. There are two problems here.

     1. My understanading of the NODE[$token] leads me to believe that this
should be a createTypeToken call instead of a createTypeText call.
     2. Even otherwise, casting retval which is a rule return type to a
token is a bug and introduces a compilation error.

I would appreciate it very very much if someone could help me fix this. I am
hoping a fix is needed in the AST.stg file packed in with the jar.

It is entirely likely that I am doing something stupid here (I hope not as I
pored over the book multiple times) and coud do with some expert advice.

Either way, appreciate a reply.

thanks very much.

-vamsi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20081102/0fdf50b0/attachment.html