[antlr-interest] Preserving Original Token Sequence In ASTs

Terence Parr parrt at cs.usfca.edu
Fri Sep 5 18:17:35 PDT 2003


Damn!  The stupid URL wrapped...careful when you click on it ;)

Ter

On Friday, September 5, 2003, at 06:14 PM, Terence Parr wrote:

> Guys,
>
> Had a chat with Greg Benson (an OS maniac and fellow professor here at
> USF) yesterday and he described some transformations he had done in
> JavaCC to build efficient parallel processing libraries (special
> encodings for various types etc...).  He had a nice translator solution
> where it was easy to get the original token order from his trees so he
> could do easy Java->Java transformations.  We talked about the
> difference between parse trees and ANTLR's ASTs and how ASTs are what
> you want for translation, but it's hard to get the original token
> sequence back out.  I decided to tackle the issue today and have posted
> an article describing my results.  Thanks for the motivation, Greg!  We
> may have the best of both worlds now. :)
>
> Here is what I conjured up and just posted to antlr.org:
>
> http://www.antlr.org/article/preserving.token.order/
> preserving.token.order.tml
>
> "You want ASTs instead of parse trees for translation, but ASTs have
> the token order shuffled from the one-dimensional input sequence into a
> two-dimensional tree structure.  Consequently, it is often hard to spit
> the original token sequence back out.  You are required to build a tree
> walker that spits text back out.  This article describes an
> experimental means of supporting easy x to x translation without
> modifying ANTLR itself.  I provide modified token, token stream, tree
> node, and AST factory objects along with a simple grammar to
> demonstrate their use.  This is damn cool stuff and will be standard
> equipment on ANTLR 3."
>
> Let me know what you think. :)  The solution is cute if ya ask me. ;)
>
> L8R G8Rs,
> Terence
> --
> Professor Comp. Sci., University of San Francisco
> Creator, ANTLR Parser Generator, http://www.antlr.org
> Co-founder, http://www.jguru.com
> Co-founder, http://www.knowspam.net enjoy email again!
> Co-founder, http://www.peerscope.com pure link sharing
>
>
>
>
>
>
> Your use of Yahoo! Groups is subject to 
> http://docs.yahoo.com/info/terms/
>
>
>
--
Professor Comp. Sci., University of San Francisco
Creator, ANTLR Parser Generator, http://www.antlr.org
Co-founder, http://www.jguru.com
Co-founder, http://www.knowspam.net enjoy email again!
Co-founder, http://www.peerscope.com pure link sharing




 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 




More information about the antlr-interest mailing list