[antlr-interest] code translation

[Benoit Fouletier]

Thanks James!
Yes, I'm going the "rewrite the whole grammar" route. I'd really like to use
v3 though (to benefit from ANTLRWorks), and I'm really struggling to sort
out through all the v2 info... a LOT has changed and doesn't apply, and
converting is a mess if you don't know both v2 and v3 really well.

Anyway... the translation itself is relatively straightforward, ANTLR is
amazing for that!, but preserving comments & formatting is killing me. The
best resources I found were all in v2, and much seems to have changed
(the token
stream filtering/splitting <http://www.antlr2.org/doc/streams.html>, and the
"hidden channel" helpers are gone). I understand v3 is supposed to be more
generic & flexible (multiple channels instead of just "hidden or not"), but
I just don't understand what the intended way is for a translation scenario.

        Ben


On Thu, Jul 2, 2009 at 5:38 PM, James Abley <james.abley at gmail.com> wrote:

> 2009/6/24 Benoit Fouletier <benblo+ANTLR at gmail.com<benblo%2BANTLR at gmail.com>
> >:
> > Hi!
> > Newbie to ANTLR here, hello all =).
> > I'm trying to make a translator from Unity's JavaScript (sometimes called
> > UnityScript) to C#. This particular JS implementation is made to compile
> to
> > IL and run on Mono. It's actually based on Boo, the grammar was written
> > by Rodrigo B. de Oliveira with ANTLR (don't know which version).
> > So the 2 languages are very similar, they don't differ in principle, only
> in
> > syntax. I've already achieved a lot with regular expressions but that
> just
> > doesn't seem like the right tool for the job.
> > 90% of the differences are in variable & function declaration, the core
> is
> > the same.
> > Basically, I need to translate this:
> >
> > function MyFunction(i : int, b : boolean) : String
> > {
> >     return i + " " + b;
> > }
> >
> > to this:
> >
> > private string MyFunction(int i, boolean b)
> > {
> >     return i + " " + b;
> > }
> >
> > I have access to the compiled lexer & parser (but not the original
> grammar).
> > I've tried to go through the token stream and manually rearrange the
> > declarations (which works but again, doesn't feel like the best way to do
> > it).
> > The thing is, I want to retain comments & formatting during translation,
> but
> > the lexer doesn't give me the hidden tokens* (and anyway my little
> fiddlings
> > would probably explode!).
> > Also, I don't need a complete compiler: all the code I want to translate
> > already compiles so I know it's valid.
> >
> > It seems to me like my needs are fairly simple (!), but I don't know what
> > approach to take. If I were to rewrite the grammar (or just modify the
> > ECMAScript grammar found here), are there ways to define translate rules,
> in
> > the fashion of regex replacement? Or should I be able to get away with
> using
> > the compiled lexer I have?
> >
> > Sorry my post is so long, I hope I've made the context clear. Cheers =),
> >
> >         Ben
> >
> > * I've read the token streams article in the v2 docs (it's not in the v3
> > docs, is it still valid?), which is very informative... unfortunately
> when I
> > setTokenCreator to a hidden stream, I don't get anything more.
> > Here's my loop:
> > UnityScriptLexer lexer = new UnityScriptLexer(file);
> > lexer.setTokenCreator(new
> > antlr.CommonHiddenStreamToken.CommonHiddenStreamTokenCreator());
> >
> > antlr.IHiddenStreamToken token;
> > while ((token = (antlr.IHiddenStreamToken)lexer.nextToken()).Type != EOF)
> > {
> >     if (token.getHiddenBefore() != null)
> >         LogToken(token.getHiddenBefore());   // this never happens
> >
> >     LogToken(token);
> >
> >     if (token.getHiddenAfter() != null)
> >         LogToken(token.getHiddenAfter());    // this never happens
> > }
> > LogToken() just outputs to the console.
> >
> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > Unsubscribe:
> > http://www.antlr.org/mailman/options/antlr-interest/your-email-address
> >
>
>
> http://www.antlr.org/wiki/display/ST/Language+Translation+Using+ANTLR+and+StringTemplate
>
> That seems to be ANTLR 2. but relevant. You would need a grammar
> though, rather than just the compiled lexer and parser.
>
> The new book is supposed to contain information around this sort of
> request. Ter might be able to tell more about that.
>
> http://www.pragprog.com/titles/tpdsl/language-design-patterns
>
> Cheers,
>
> James
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20090702/f443f342/attachment.html