[antlr-interest] Duplicating and modifying portions of the input

[Geraint North]

Hi,

First off, I'd like to thank Terence and anyone else involved in the  
development of ANTLR/ANTLRWorks - I've just picked up ANTLR (v3),  
having looked at ANTLR last back in 2002 (if memory serves me) and I'm  
finding it an absolute joy to use.

I was wondering if I could get some suggestions from the group as to  
how to implement a particular feature in a language I'm developing.

The majority of the language syntax is unimportant, but it is vaguely  
C-like in its language and comment style, with the only major  
difference is the use of a period (".") character within function  
names, as shown in this declaration of the "foo.32" function:

void foo.32( int a )
{
/*
   This is a multi-line comment
*/
   do.thing.32(a);   // Single-line comment, call to the "do.thing.32"  
function.
}

I have a templating mechanism that can be used to shorthand several  
similar methods - hopefully obvious in it's use:

template { SIZE=8,16,32,64 }
void foo.SIZE( int a )
{
  /*
    This is a multi-line comment
  */
   do.thing.SIZE(a);    // Single-line comment
}

The above template code is equivalent to defining four methods, "foo. 
8", "foo.16", "foo.32" and "foo.64", each one calling the appropriate  
"do.thing.*" method.

I currently implement this by parsing the source with ANTLR, and then  
manually walking the AST, performing a deep-copy on function  
declarations found within templates, substituting the symbols where  
appropriate.  I can then pass the AST to a StringTemplate  
implementation to produce something readable, but comments and  
formatting are lost.

However, since the template code can get rather complex (moreso than  
the example I've provided), I would like to provide the user with the  
option to view the "expanded" source code, complete with original  
comments and formatting.

The solution clearly lies somewhere in the realm of a "Rewriter", as  
described in the ANTLR book, but I'm unsure as to how much of my task  
I can automate, and how much I need to do by hand.

It seems that if the simple deep copy of the AST that I am performing  
would also preserve the hidden tokens (does it?), then I should just  
be able to feed to resulting expanded tree into a "null" rewriter, and  
that should produce the expanded test, comments and all - is that  
correct, or do I have to do something to ensure that the token stream  
is also deep copied?

My simple deep copy is just:

public static CommonTree deepCopy( CommonTree tree )
   {
     CommonTree res = (CommonTree)tree.dupNode();
     for ( int i = 0; i < tree.getChildCount(); i++ )
     {
       res.addChild( deepCopy( (CommonTree)tree.getChild(i) ) );
     }
     return res;
   }


Thanks,
Geraint North.