[antlr-interest] New article on StringTemplates and Treewalke rs

[Nagesh, Harsha]

Andy,

	From what you describe and from a bit of Stratego that I know, I think
you maybe doing something that Stratego supports very well (atleast the part
of listing-rules-and-firing-them-without-embedding-them-inside-grammar)
Stratego has constructs to specify pattern matching rules and also has
a library of calls which direct how a rule can be applied. For example,
you can specify that a rule can be applied top-down on the tree and where
ever there is a match for the pattern of the rule, the transformation is
applied and there are many other in built "strategies" (as stratego calls
it) for transformation. However, my experience with stratego has been that
it has a steep learning curve, poorly documented and does not work for
larger real world examples (or atleast the knowledge of making it work
for such problems is buried deep in the minds of the developers of 
stratego - who are grad students with paper deadlines ! ) :(

I also want to add a few words about my experience with ANTLR - it was easy to
pick up, as I could write some java/c# code when a pattern matched and
found syntax directed translation much easier to get a complex example
working than a AST based transformation. I find that tree based Xformations
in antlr is not at all well documented and has syntax that is buried
somewhere in the manual, which has only few toy examples. I think that the
there is also no single place on antlr's website where I can systamatically
learn the differences of both the approaches, how does antlr contrast to
other systems (not by theory, but by examples) or even for that matter,
what is new in antlr v3 about which there is so much traffic on this list.
Forgive me for venting my frustration, maybe its just me and my slow
thinking, but I think if there is more well structured documentation, 
organized well on antlr.org it would be so much easier for a newbie...and
to add to this confusion more sites are being created for string template,
and there was one for programtransformation.org (?) etc...For a new person
it takes a good bit of effort to understand what is what and how do all these
tools come together. (In stratego, there have more than 50 different
executables which can be piped in a chain to get something meaningful - imagine
the nightmare for even a competant person to understand the tool linkage...
a lot of effort is spent into learning of how to impleemnt something in the
system, than the things that you want your program to do..

just my 2c

Harsha




This material has been prepared by individual sales and/or trading personnel and does not constitute investment research.  Please follow the attached hyperlink to an important disclaimer: http://www.csfb.com/legal_terms/disclaimer_americas_salestrading.shtml




-----Original Message-----
From: antlr-interest-bounces at antlr.org
[mailto:antlr-interest-bounces at antlr.org]On Behalf Of Andy Tripp
Sent: Wednesday, January 11, 2006 12:01 AM
To: antlr-interest at antlr.org
Subject: [antlr-interest] New article on StringTemplates and Treewalkers


I must be driving you guys nuts by not posting any code, so here is some.

On the issue of that we stumbled on of checking for "if (x=5)", here's 
what I
do. I have a ForceToBooleanRule that checks for all the places that
Java requires a boolean expression. At each of those, I use ANTLR to
parse the expression into an AST, and then traverse that tree, changing 
things
from int to boolean type. That's a bit vague, I know. And the 
lexer/parser/expressionChanger
 code isn't even shown here, it's within the 
ExpressionUtils.forceToBoolean() call.
What is shown here is just the "ForceToBooleanRule" code: A rule that's 
written
in plain Java code, that looks for various patterns in a Token stream 
(NOT an AST).

As always, I do think I have a point here. That is the issue of "where 
would a Rule like this be
invoked in the treewalker approach?" Do you really want to go through 
the bother
of finding the right place for this inside the ANTLR C grammar?
Is there even really a valid set of "right places" that represent the 
four bullet items
in the code below (beneath "for", "while", "do-while", and "if")?
 I don't want to bother with all that. I'd rather do what this code does...
just assume that we're looking through a sequence of Tokens, and do just 
two checks:
 Are we at a "for" (and if so, the expression is between the next ";" 
and the following ";")?
 Are we at a "while" or "if" (if so the expression is everything between 
the next "(" and its matching ")".

No AST tree structure knowledge required! Just write the rule code using
the more intuitive "sequence of tokens" mental image.

(Some background: the match() method is called on every Token in every C 
file. If
it returns true, then the apply() method is called to make the change. 
The Source class
has lots of methods for working with a sequence of Tokens.)
--------------------------------------------------------------------------------------
// certain places in Java require a boolean type, where C allows "int" type:
// * for condition:  for (int i=0; b; i++)
// * while:  while (b)
// * do-while: do { } while (b);
// * if: if (b)

public class ForceToBooleanRule extends Rule {
    private String newexpr;
    private Token startToken = null;
    private Token endToken = null;

    // if we see one of the patterns listed above, set startToken and 
endToken and return true.
    public boolean match(Source source) {
        String current = source.currentToken.getText();
        if (current.equals("for")) {
            startToken = source.findToken(";");
            endToken = source.findToken(source.getTokenAt(startToken, 
+1), ";");
        }
        else if (current.equals("while") ||
                 current.equals("if")) {
            startToken = source.getTokenAt(+1); // '('
            endToken = source.findMatchingParen(startToken);
        }
        else {
            return false;
        }
        String expr = source.getTextBetween(startToken, endToken);

        // here is where the expression is parsed and changed to be 
boolean type:
        newexpr = ExpressionUtils.forceToBoolean(expr, source);
        return !expr.equals(newexpr);
    }

    // change the expression:
    public void apply(Source source) {
        source.replaceTokenRange(startToken, endToken, 
startToken.getText()+newexpr+endToken.getText());
    }
}


==============================================================================
Please access the attached hyperlink for an important electronic communications disclaimer: 

http://www.csfb.com/legal_terms/disclaimer_external_email.shtml

==============================================================================