[antlr-interest] Preserving Original Token Sequence In ASTs

[rodrigo reyes]

Hi Terence,

> "You want ASTs instead of parse trees for translation, but ASTs have  
> the token order shuffled from the one-dimensional input 
> Let me know what you think. :)  The solution is cute if ya ask me. ;)

Very interesting. Just a few comments:

1/ I didn't understand the part which describes the differences between
parse trees and AST. Specially: "Regardless of how you express a grammar,
the AST could always remain the same". From what I've seen, this is more a
feature of the antlr parser, which allows the grammar developer to change
the AST topology at will (using ^ and #[]). Did I miss a point ?

2/ The main point of translation is that the original tree is modified. A
standard L-to-L translator is going to parse the tree, and modify the nodes:
remove this and this, add a new one, reorder this and this, etc. It is not
clear in your article what is the mechanism one should use to keep the token
stream and the AST synchronized, and it seems to be a major issue.

Bests,
Rodrigo

 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/