[antlr-interest] Transforming Comments along with trees

Terence Parr parrt at cs.usfca.edu
Thu Nov 11 08:02:32 PST 2004



On Nov 11, 2004, at 3:13 AM, dsfsdf fdsfdsf wrote:

> ya i have seen the TokenStreamRewriteEngine doc on the website, i feel 
> it would be a bit complicated to implement as the input could be huge,

Bigger than virtual memory? ;)

>  and it doesn't show how to handle comments,

They just work.  You get the whole stream of tokens but the parser 
ignores comments.  You do cut/paste operations to the buffer then say 
toString() and it comes back out with you rewrites.  no fuss, no muss!

>  besides i'm not too comfortable with it.

why not?

> As for the latter question, after each pass i generate a new tree.
> Should these passes automatically bring across the hidden tokens?
> how? why not?

If you're doing tree rewrites, then yes, they have to come along.  The 
tree input->output thing should copy that stuff automatically I think.

> And how do i output these hidden tokens after all my transformations 
> are complete?

Do a tree walk and spit them out as you spit the other stuff out.

Ter

> thanks
> ed 
>
> Terence Parr <parrt at cs.usfca.edu> wrote:
>
>
> On Nov 10, 2004, at 4:22 AM, dsfsdf fdsfdsf wrote:
>
> > Hi
> > I am building a parser that will convert a piece of t-slq to the
>  > corresponding p-sql. I am using a TokenStreamHiddenTokenFilter to
> > ignore but preserve comments and builld up my source ast tree!
> > I than plan to do maybe one or to pass transformations to the tree!
> > However i am stuck on how to bring across my comments after each
>  > transformations!
> > I would greatly appreciate any ideas on how this can be done!
>
> Hi. First, given the similarity of source / target languages, you
>  might want to use TokenStreamRewriteEngine instead. have you seen that
>  article on the site?
>
> For the above, are you saying that the auto construction of trees in
> the tree walker doesn't carry along the hidden stream stuff?
>
> Ter
> --
> CS Professor &am p; Grad Director, University of San Francisco
> Creator, ANTLR Parser Generator, http://www.antlr.org
> Cofounder, http://www.jguru.com
> Cofounder, http://www.knowspam.net enjoy email again!
>
>
>
>
>
>
> Yahoo! Groups Links
>
>
>
>
>
>
>
>
> ALL-NEW Yahoo! Messenger - all new features - even more fun!
>
>  Yahoo! Groups Links
>
> 	• 	To visit your group on the web, go to:
> http://groups.yahoo.com/group/antlr-interest/
>  
> 	• 	 To unsubscribe from this group, send an email to:
> antlr-interest-unsubscribe at yahoogroups.com
>  
> 	• 	 Your use of Yahoo! Groups is subject to the Yahoo! Terms of 
> Service.
>
>  




 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
    antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 





More information about the antlr-interest mailing list