[antlr-interest] code translation

[Terence Parr]

On Jul 2, 2009, at 9:43 AM, Benoit Fouletier wrote:
> Anyway... the translation itself is relatively straightforward,  
> ANTLR is amazing for that!, but preserving comments & formatting is  
> killing me. The best resources I found were all in v2, and much  
> seems to have changed (the token stream filtering/splitting, and the  
> "hidden channel" helpers are gone).

Really?  $channel = HIDDEN doesn't work in lexer?

Ter
>  I understand v3 is supposed to be more generic & flexible (multiple  
> channels instead of just "hidden or not"), but I just don't  
> understand what the intended way is for a translation scenario.
>
>         Ben
>
>
> On Thu, Jul 2, 2009 at 5:38 PM, James Abley <james.abley at gmail.com>  
> wrote:
> 2009/6/24 Benoit Fouletier <benblo+ANTLR at gmail.com>:
> > Hi!
> > Newbie to ANTLR here, hello all =).
> > I'm trying to make a translator from Unity's JavaScript (sometimes  
> called
> > UnityScript) to C#. This particular JS implementation is made to  
> compile to
> > IL and run on Mono. It's actually based on Boo, the grammar was  
> written
> > by Rodrigo B. de Oliveira with ANTLR (don't know which version).
> > So the 2 languages are very similar, they don't differ in  
> principle, only in
> > syntax. I've already achieved a lot with regular expressions but  
> that just
> > doesn't seem like the right tool for the job.
> > 90% of the differences are in variable & function declaration, the  
> core is
> > the same.
> > Basically, I need to translate this:
> >
> > function MyFunction(i : int, b : boolean) : String
> > {
> >     return i + " " + b;
> > }
> >
> > to this:
> >
> > private string MyFunction(int i, boolean b)
> > {
> >     return i + " " + b;
> > }
> >
> > I have access to the compiled lexer & parser (but not the original  
> grammar).
> > I've tried to go through the token stream and manually rearrange the
> > declarations (which works but again, doesn't feel like the best  
> way to do
> > it).
> > The thing is, I want to retain comments & formatting during  
> translation, but
> > the lexer doesn't give me the hidden tokens* (and anyway my little  
> fiddlings
> > would probably explode!).
> > Also, I don't need a complete compiler: all the code I want to  
> translate
> > already compiles so I know it's valid.
> >
> > It seems to me like my needs are fairly simple (!), but I don't  
> know what
> > approach to take. If I were to rewrite the grammar (or just modify  
> the
> > ECMAScript grammar found here), are there ways to define translate  
> rules, in
> > the fashion of regex replacement? Or should I be able to get away  
> with using
> > the compiled lexer I have?
> >
> > Sorry my post is so long, I hope I've made the context clear.  
> Cheers =),
> >
> >         Ben
> >
> > * I've read the token streams article in the v2 docs (it's not in  
> the v3
> > docs, is it still valid?), which is very informative...  
> unfortunately when I
> > setTokenCreator to a hidden stream, I don't get anything more.
> > Here's my loop:
> > UnityScriptLexer lexer = new UnityScriptLexer(file);
> > lexer.setTokenCreator(new
> > antlr.CommonHiddenStreamToken.CommonHiddenStreamTokenCreator());
> >
> > antlr.IHiddenStreamToken token;
> > while ((token = (antlr.IHiddenStreamToken)lexer.nextToken()).Type ! 
> = EOF)
> > {
> >     if (token.getHiddenBefore() != null)
> >         LogToken(token.getHiddenBefore());   // this never happens
> >
> >     LogToken(token);
> >
> >     if (token.getHiddenAfter() != null)
> >         LogToken(token.getHiddenAfter());    // this never happens
> > }
> > LogToken() just outputs to the console.
> >
> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > Unsubscribe:
> > http://www.antlr.org/mailman/options/antlr-interest/your-email-address
> >
>
> http://www.antlr.org/wiki/display/ST/Language+Translation+Using+ANTLR+and+StringTemplate
>
> That seems to be ANTLR 2. but relevant. You would need a grammar
> though, rather than just the compiled lexer and parser.
>
> The new book is supposed to contain information around this sort of
> request. Ter might be able to tell more about that.
>
> http://www.pragprog.com/titles/tpdsl/language-design-patterns
>
> Cheers,
>
> James
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20090702/5a137865/attachment.html