[antlr-interest] [C target] [3.1.1] Deriving imaginary nodes from real tokens

Sven Van Echelpoel sven.van.echelpoel at empolis.com
Thu Dec 11 06:17:58 PST 2008


Hi,

According to the book you can derive imaginary nodes from real tokens to
take over attributes (such as line number) from the latter. The syntax
for this is :

foo : b='bar' -> ^( BAR[ $b ] )
    ;

However, when you do this with the C target, no line numbers or anything
gets set. After some sleuthing I found the templates in AST.stg that
might be relevant to this. They look like this:

createImaginaryNode(tokenType,hetero,args) ::= <<
<if(hetero)>
<! new MethodNode(IDLabel, args) !>
<hetero>New(<tokenType><if(args)>, <args; separator=", "><endif>)
<else>
<if(args)>

#if <length(args)> == 2
	(<ASTLabelType>)ADAPTOR->createTypeTokenText(ADAPTOR, <tokenType>,
TOKTEXT(<args; separator=", ">))
#else
	(<ASTLabelType>)ADAPTOR->createTypeText(ADAPTOR, <tokenType>,
(pANTLR3_UINT8)<args; separator=", ">)
#endif

<else>
(<ASTLabelType>)ADAPTOR->createTypeText(ADAPTOR, <tokenType>,
(pANTLR3_UINT8)"<tokenType>")
<endif>
<endif>
>>

createRewriteNodeFromElement(token,hetero,args) ::= <<
<if(hetero)>
<hetero>New(stream_<token>->nextToken(stream_<token>)<if(args)>, <args;
separator=", "><endif>)
<else>
<if(args)> <! must create new node from old !>

#if	<length(args)> == 2
(<ASTLabelType>)ADAPTOR->createTypeTokenText(ADAPTOR,
<token>->getType(<token>, TOKTEXT(<token>, <args; separator=", ">)) /*
JIMI */
#else
ADAPTOR->createTypeToken(ADAPTOR, <token>->getType(<token>, <token>,
<args; separator=", ">)
#endif

<else>
stream_<token>->nextNode(stream_<token>)
<endif>
<endif>
>>

In the first one, createImaginaryNode, if you pass one argument to it,
which I'm assuming to be the case, a type from a text is created. In the
second one, createRewriteNodeFromElement, a type from a token is created
with the same number of arguments.

Although I have not dug any further, I can tell from examining the
generated parser code that in an AST rewrite, the createImaginaryNode is
used, resulting in nothing (the cast to pANTLR3_UINT8 probably and
luckily makes it point to a zero, immediately terminating the string).

It can be made to work by passing the second argument (the text), even
though I have no use for that.

foo : b='bar' -> ^( BAR[ $b, "" ] )
    ;

Am I missing something, or should that normally work as advertised? 

Sven



More information about the antlr-interest mailing list