[antlr-interest] De-emphasizing tree grammars?

[Terence Parr]

Hi George,

I'm providing AST construction and tree grammars for advanced users and for backward compatibility. I have done some tree rewriters for the book, yes, but I have found that they work well only for such things as peephole optimization.   The AST/ tree grammar mechanism also is something I've been pushing for 20 years without success.  I started out in the compilers world where you really do need an AST, to avoid all of the noise of rule names inside your tree. Recently I had the chance to reevaluate and try out a simple new mechanism based upon parse trees, which can be automatically generated. Now, I'm doing both a DOM and a SAX event type model, which is suitable for lots of different applications. For example, if you want to collect information about all Java declarations, you can simply pass in a listener to the Java parser. The parser doesn't have to be one that happens to have AST construction rules.

Anyway, this is just a taste of what I've been thinking about and it definitely seems to fit within the marketplace / common tasks better. While v4 will have lots of great new features, if you have something working with v3, I'm not sure there is a huge reason to convert. v4 is a lot easier to build grammars and simpler to use in terms of language application implementation, but if you already have a grammar that works, It's probably not worth the effort to try to upgrade. In the case of tree rewriting grammars, if they work, I would just leave them in v3. You will be able to mix the two (v3, v4) freely given the different package names.

Ter


On Dec 19, 2011, at 8:46 AM, George S. Cowan wrote:

> I am worried about my use extensive use of tree grammars in my project.

> On the TODO List page
> http://www.antlr.org/wiki/display/~admin/TODO+list?focusedCommentId=27656324
> #comment-27656324
> 
> Ruben Laguna says:
>> Can I ask why the "no ast output from tree grammar" bullet? Is there
> something fundamentally wrong with generating ast from tree grammars?
> 
> then
> 
> Terence Parr says:
>> Hi Ruben, turns out I'm de-emphasizing tree grammars and it's a rarely
> used feature and hard to implement.
> 
> That "de-emphasizing tree grammars" is really scary. The TODO list still
> contains some items about tree grammars, TreeFilter, and TreeRewriter so I'm
> not clear. Also, in the Language Implementation Patterns book, the tree
> grammar examples Reduce.g and Simplify.g use output=AST. You seemed very
> excited about that capability not so long ago. 
> 
> Will I still be able to do extensive transformations of trees in ANTLR 4?
> What approach should I plan on for multiple complicated outputs from a
> single input?
> 
> George
> 
> 
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address