[antlr-interest] How to copy/reference hidden tokens in transformations

[dsfsdf fdsfdsf]

hi
No bug existed,error existed in my tree walker code
thanks
ed.

dsfsdf fdsfdsf <eddie_antlr at yahoo.co.uk> wrote:
hi ter
There may be one other glitch,
I have a file ASTPass1 which transforms one of my trees,
However antlr does not generate this tree with the comments.
I have gone through the generated ASTPass1.java file and 
have found that the root of this problem lies within the 
ASTFactory method >
 
public AST make(AST[] nodes)
 
This is a sample of my ASTPass1 file
 
ASTArray ASTarray = (new ASTArray(5)).add(tmp1_AST).add(s_AST).add(into_tree).add(f_AST).add(w_AST);
  
  Parser.treeWalker(((CommonASTWithHiddenTokens2)ASTarray.array[3]));

  select_AST=(CommonASTWithHiddenTokens2)astFactory.make(ASTarray); 
  Parser.treeWalker((CommonASTWithHiddenTokens2)select_AST);
 
I do a tree walk before the ASTFactory.make method gets called and here i input the AST that has the hidden token and it correctly outputs the hidden token.
However when i do a tree walk after the make() methos gets called
the hidden token doesn't get printed out.
I have debugged through the ASTFactory.make method and i can see that the AST that is supposed to hold the token does exist and the hidden token exists.
Maybe its worth noting that the AST that has the hidden token has a firstChild of a normal token which in turn is linked to the hidden token.
 
Does the ASTFactory class need to be overidden maybe?
 
thanks
ed.


Terence Parr <parrt at cs.usfca.edu> wrote:

Awesome. Fixed in main line code.
Ter
On Nov 17, 2004, at 6:28 AM, dsfsdf fdsfdsf wrote:

> hi ter
> Did that suggestion and it worked.
>  
> public void initialize(AST t)
> {  hiddenBefore = ((CommonASTWithHiddenTokens)t).getHiddenBefore();
>    hiddenAfter = ((CommonASTWithHiddenTokens)t).getHiddenAfter();
>    super.initialize(t) 
> }
>  
> Both hiddenBefore and hiddenAfter must be cast to type 
> CommonASTWithHiddenTokens to allow access to the methods 
> getHiddenAfter() and getHiddenBefore()
> And replace CommonASTWithHiddenTokens objects with your new class in 
> your parser class.
> thanks for all the help.
> ed. 
>  
>
> Terence Parr 
wrote:
>
>
> On Nov 16, 2004, at 8:53 AM, dsfsdf fdsfdsf wrote:
>
> > When i am creating a new tree from my source tree that has some 
> hidden
> > tokens, how do i copy these hidden across?
> > At the moment it is not doing it automatically.
> > thanks
>
> Hi Ed. Can't remember so I did a test.
>
> Here is a grammar:
>
> class TP extends TreeParser;
>
> options {
> buildAST=true;
> }
>
> a : ID ;
>
> It generates relevant code:
>
> tmp1_AST = astFactory.create((AST)_t);
> tmp1_AST_in = (AST)_t;
> astFactory.addASTChild(currentAST, tmp1_AST);
> match(_t,ID);
>
> So I looked at:
>
> /** Create a new empty AST node; if the user did not specify
> * an AST node type, then create a default one: CommonAST.
> */
> public AST create(AST tr) {
> if (tr == null) return null; // create(null) == null
> AST t = create(tr.getType());
> if ( t !=null ) {
> t.initialize(tr);
> }
> return t;
> }
>
> which invokes CommonASTWithHiddenTokens.initialize(AST) but that is
> inherited from CommonAST:
>
> public void initialize(AST t) {
> setText(t.getText());
> setType(t.getType());
> }
>
> Hmm...seems like that should be override just like I do for Token arg:
>
> public void initialize(Token tok) {
> CommonHiddenStreamToken t = (CommonHiddenStreamToken)tok;
> super.initialize(t);
> hiddenBefore = t.getHiddenBefore();
> hiddenAfter = t.getHiddenAfter();
> }
>
> Seems I forgot to do this. Ok, I suggest subclassing
> CommonASTWithHiddenTokens and overriding to be something like:
>
> public void initialize(AST t) {
> hiddenBefore = t.getHiddenBefore();
> hiddenAfter = t.getHiddenAfter();
> super.initialize(t);
> }
>
> Let me know if that works and I'll put it in ;)
>
> Don't forget to use the new subclass when you set the AST type etc...
> in the parser/tree parser.
>
> Ter
> --
> CS Professor & Grad Director, University of San Francisco
> Creator, ANTLR Parser Generator, http://www.antlr.org
> Cofounder, http://www.jguru.com
> Cofounder, http://www.knowspam.net enjoy email again!
>
>
>
>
>
>
> Yahoo! Groups Links
>
>
>
>
>
>
>
>
> Win a castle for NYE with your mates and Yahoo! Messenger
>
> Yahoo! Groups Links
>
> • To visit your group on the web, go to:
> http://groups.yahoo.com/group/antlr-interest/
>  
> • To unsubscribe from this group, send an email to:
> antlr-interest-unsubscribe at yahoogroups.com
>  
> • Your use of Yahoo! Groups is subject to the Yahoo! Terms of 
> Service.
>
>
--
CS Professor & Grad Director, University of San Francisco
Creator, ANTLR Parser Generator, http://www.antlr.org
Cofounder, http://www.jguru.com
Cofounder, http://www.knowspam.net enjoy email again!






Yahoo! Groups Links









---------------------------------
Win a castle for NYE with your mates and Yahoo! Messenger 

---------------------------------
Yahoo! Groups Links

   To visit your group on the web, go to:
http://groups.yahoo.com/group/antlr-interest/
  
   To unsubscribe from this group, send an email to:
antlr-interest-unsubscribe at yahoogroups.com
  
   Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service. 


		
---------------------------------
 ALL-NEW Yahoo! Messenger - all new features - even more fun!  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20041123/37bf4ddf/attachment.html