[antlr-interest] How to copy/reference hidden tokens in transformations

Terence Parr parrt at cs.usfca.edu
Tue Nov 16 09:59:39 PST 2004



On Nov 16, 2004, at 8:53 AM, dsfsdf fdsfdsf wrote:

> When i am creating a new tree from my source tree that has some hidden 
> tokens, how do i copy these hidden across?
> At the moment it is not doing it automatically.
> thanks

Hi Ed.  Can't remember so I did a test.

Here is a grammar:

class TP extends TreeParser;

options {
         buildAST=true;
}

a : ID ;

It generates relevant code:

                         tmp1_AST = astFactory.create((AST)_t);
                         tmp1_AST_in = (AST)_t;
                         astFactory.addASTChild(currentAST, tmp1_AST);
                         match(_t,ID);

So I looked at:

     /** Create a new empty AST node; if the user did not specify
      *  an AST node type, then create a default one: CommonAST.
      */
     public AST create(AST tr) {
         if (tr == null) return null;		// create(null) == null
         AST t = create(tr.getType());
		if ( t!=null ) {
			t.initialize(tr);
		}
         return t;
     }

which invokes CommonASTWithHiddenTokens.initialize(AST) but that is 
inherited from CommonAST:

     public void initialize(AST t) {
         setText(t.getText());
         setType(t.getType());
     }

Hmm...seems like that should be override just like I do for Token arg:

     public void initialize(Token tok) {
         CommonHiddenStreamToken t = (CommonHiddenStreamToken)tok;
         super.initialize(t);
         hiddenBefore = t.getHiddenBefore();
         hiddenAfter = t.getHiddenAfter();
     }

Seems I forgot to do this.  Ok, I suggest subclassing 
CommonASTWithHiddenTokens and overriding to be something like:

     public void initialize(AST t) {
         hiddenBefore = t.getHiddenBefore();
         hiddenAfter = t.getHiddenAfter();
         super.initialize(t);
     }

Let me know if that works and I'll put it in ;)

Don't forget to use the new subclass when you set the AST type etc... 
in the parser/tree parser.

Ter
--
CS Professor & Grad Director, University of San Francisco
Creator, ANTLR Parser Generator, http://www.antlr.org
Cofounder, http://www.jguru.com
Cofounder, http://www.knowspam.net enjoy email again!





 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
    antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 





More information about the antlr-interest mailing list