[antlr-interest] philosophy about translation
Andy Tripp
antlr at jazillian.com
Fri Oct 6 13:15:27 PDT 2006
Terence Parr wrote:
>
> On Oct 6, 2006, at 6:59 AM, Andy Tripp wrote:
>
>> Here's my alternative:
>> I have hundreds of "transformation rules" where each rule can either
>> be a simple mapping like this:
>> ADD v1 TO v2. --> v2 += v1;
>> ADD v1 v2 TO v3 v4. --> v3 += v1 + v2; v4 += v1 + v2
>> ADD v1 TO v2 GIVING v3. --> v3 = v1 + v2;
>>
>> ...or actual Java code that operates on sequences of tokens, not trees.
>> I find it easier to think of the source and target languages as
>> sequences of tokens
>> ("ADD" followed by a variable followed by "TO"...) rather than trees
>> (an "ADD" node
>> with the first child being a variable and the second child being
>> "TO"...)
>
>
> interesting. What happens when you need to use structure not just
> token sequence?
I have a large library of methods that work on token streams. Methods like:
getContainingFunction, getNextScope, findNextToken("foo"), etc. As for
"knowing where you are in the tree", such as "look for this pattern of
function declaration
(but of course don't look for it *within* another function)", I just
don't do that. i.e.
I might look for pattern "f(...) {" anywhere.
>
> how did you decide that you could replace those variables with
> strings? Did you do flow analysis? If so how did you do it on a
> linked list of tokens?
Actually, in this case, now that I think about it, I lied...I would generate
System.out.println(hello + world);
But In some places, I do look at all references to a variable, see that
none of them are setting the value, and so make it
"final static" if it's at the class level, and sometimes get rid of the
variable declaration altogether. Ad hoc, I know :(
>
>> I guess what I'm doing is more akin to natural language translation.
>> I'm sure an English to
>> Spanish translator has relatively little logic that does tree
>> transformation (e.g. putting the
>> adjectives *after* the nouns).
>
>
> an interesting analogy. You are saying that language translation is
> mostly word for word change and some grammatical changes such as you
> would see from English to German where the verb's at the end.
I had spent just as much time investigating natural language processing
systems as programming language processing systems
when I started. My impression is that all the work in NPL is in
understanding context and handling exceptions, not word
replacement. "white house" becomes "casa blanca" (not "casa blanco").
But "The White House" in Spanish is still "The White House"!
I saw an poster for a NLP conference once and I noted that a newpaper
headline in the back said "Woods Eyes Masters".
Talk about context! Try parsing that sentence if it's not the headline
of a sports page!
>
>
> Hmm.... Do you have multiple phases also, just declarative rule
> replacement style?
Not sure what you mean. I have a couple hundred "rules", each really a
"phase". Each rule is either
simple declarative or actual Java code.
>
>
> Ter
>
More information about the antlr-interest
mailing list