[antlr-interest] philosophy about translation

Andy Tripp antlr at jazillian.com
Fri Oct 6 13:15:27 PDT 2006


Terence Parr wrote:

>
> On Oct 6, 2006, at 6:59 AM, Andy Tripp wrote:
>
>> Here's my alternative:
>> I have hundreds of "transformation rules" where each rule can  either 
>> be a simple mapping like this:
>> ADD v1 TO v2. --> v2 += v1;
>> ADD v1 v2 TO v3 v4. --> v3 += v1 + v2; v4 += v1 + v2
>> ADD v1 TO v2 GIVING v3.  --> v3 = v1 + v2;
>>
>> ...or actual Java code that operates on sequences of tokens, not  trees.
>> I find it easier to think of the source and target languages as  
>> sequences of tokens
>> ("ADD" followed by a variable followed by "TO"...) rather than  trees 
>> (an "ADD" node
>> with the first child being a variable and the second child being  
>> "TO"...)
>
>
>  interesting.  What happens when you need to use structure not just  
> token sequence?

I have a large library of methods that work on token streams. Methods like:
getContainingFunction, getNextScope, findNextToken("foo"), etc. As for
"knowing where you are in the tree", such as "look for this pattern of 
function declaration
(but of course don't look for it *within* another function)", I just 
don't do that. i.e.
I might look for pattern "f(...) {" anywhere.

>
>  how did you decide that you could replace those variables with  
> strings?  Did you do flow analysis?  If so how did you do it on a  
> linked list of tokens?

Actually, in this case, now that I think about it, I lied...I would generate
System.out.println(hello + world);
But In some places, I do look at all references to a variable, see that 
none of them are setting the value, and so make it
"final static" if it's at the class level, and sometimes get rid of the 
variable declaration altogether. Ad hoc, I know :(

>
>> I guess what I'm doing is more akin to natural language  translation. 
>> I'm sure an English to
>> Spanish translator has relatively little logic that does tree  
>> transformation (e.g. putting the
>> adjectives *after* the nouns).
>
>
>  an interesting analogy.  You are saying that language translation  is 
> mostly word for word change and some grammatical changes such as  you 
> would see from English to German where the verb's at the end.

I had spent just as much time investigating natural language processing 
systems as programming language processing systems
when I started. My impression is that all the work in NPL is in 
understanding context and handling exceptions, not word
replacement. "white house" becomes "casa blanca" (not "casa blanco"). 
But "The White House" in Spanish is still "The White House"!
I saw an poster for a NLP conference once and I noted that a newpaper 
headline in the back said "Woods Eyes Masters".
Talk about context! Try parsing that sentence if it's not the headline 
of a sports page!

>
>
> Hmm....  Do you have multiple phases also, just declarative rule  
> replacement style?

Not sure what you mean. I have a couple hundred "rules", each really a 
"phase". Each rule is either
simple declarative or actual Java code.

>
>
> Ter
>



More information about the antlr-interest mailing list