[antlr-interest] philosophy about translation

[Andy Tripp]

Anthony W. Youngman wrote:

> In message <4548F943.7070906 at jazillian.com>, Andy Tripp 
> <antlr at jazillian.com> writ
> I wouldn't describe "popular" as "great".

I would say popularity is a pretty good indicator or "greatness". By 
"popular", I mean "widely seen
as good from a technology point of view". So Java is "popular" that way: 
a lot of people use it
because of its technical merits. COBOL is widely used, but almost no one 
uses it because of its
technical merits.

>> Well, I think Java is far better than C++, and I think the vast 
>> majority of programmers who know both
>> agree with me. Probably the main benefit is that it's easy to use for 
>> "average programmers". That's also why
>> ANTLR is better than the competition - because it's easier to use.
>>
> You've just said Java doesn't have pointers. So how do you access a 
> hardware register in Java? How do you access a screen buffer?

You access them indirectly. Leave the underlying details to a lower layer.

>> Yes, a few people want to add stuff back, but most do not. It's just 
>> that the few are very vocal.
>> The vast majority don't want MI, operator overloading, or built-in 
>> AspectOrientedDesign.
>> And of course, those who want them think they're "above average", but 
>> then so does almost
>> everyone.
>>
> What about those who don't WANT those things, but NEED them?

Every time I've heard anyone say they NEED MI, operator overloading, or 
Aspects, I've thought
that they really don't need them, they're just not good enough 
programmers to see a clean way
to do without them. I've been programming for 25 years without ever 
NEEDING any of these,
and so have most other programmers.

>
>>>
>>>> And that's why Java is popular and Smalltalk and LISP are not.
>>>> It's also why people prefer Java over C++.
>>>>
>>>
>>> It is a easier tool to use. Less powerful. But easier.
>>>
>> Right - so I hope there's nothing wrong with me pushing to make ANTLR 
>> (or some successor)
>> easier to use. A compiler is easier to use than compiling by hand, 
>> but also less powerful.
>> I'm ok with that.
>>
> So - you're quite happy to see Antlr crippled to suit you, ignoring 
> the NEEDS of those who need its power!

Don't worry, I can't cripple ANTLR.  But yes, I'd be happy to see ANTLR4 
become a "crippled" version,
in the same sense that Java is a "crippled" version of C++.

>>>
>>>
>> But required knowledge of the tool's internals limits the "average" 
>> user's productivity.
>> If I had done AST-based translation, I'd be spending way to much time 
>> worrying about the details
>> of the AST, rather than the syntax and semantics of the two 
>> languages. I demand to spend 95% of
>> my time on *what* to do, rather than *how* to do it. With ASTs, I 
>> found myself spending
>> 95% of my time on *how*.
>>
> In other words, as you said earlier, ASTs are the wrong tool for you. 
> So you seem happy to delete ASTs from Antlr because *you* don't need 
> them, irregardless of what other people *NEED*.

Yes, just as I railed against operator overloading and MI in C++, and 
found bliss in Java.
I don't much care that other people think they *NEED* MI. They can 
always stick with C++.

>>>
>> Right, so you're just like the rest of us. You've chosen to limit 
>> your own "power" by using Java rather
>> than, say, assembly. So I'm sticking with my claim that
>> "I think a tool can be great while being simple enough for most 
>> programmers (e.g. Java)."
>> and not buying your "Not without limiting it's power" reply.
>>
> To my mind, that's not a "great" tool. As I said above, "popular", maybe.

Right, so let's just agree to disagree on that. If we can't agree that 
Java is "greater" than assembly,
we won't agree on much else.

>
> Let's ask a question ... how can a tool be "great" if it *relies* on 
> other tools even for its existence?

Sure, of course! Virtually all software relies on other tools 
(compilers, operating systems, etc) for their existence.

>
> A C compiler can compile itself. Can a Java system build itself? How 
> much of Java is actually written in Java? (Oh and I'm including the 
> supporting libraries here!)

Almost all of Java is written in Java. The only part that's not is the 
lowest layer, which is OS and hardware specific.

>
> Antlr v3 is due to be rewritten in Antlr v3. To my mind, that's a 
> "necessary but not sufficient" condition for greatness.

Wow. Only software that's written in itself can be "great"? Are you 
really saying that?

>>>
>>>
>>> He _is_ using a DSL you created to encode source-to-source 
>>> transformations.
>>> You just expressed the opinion that he has aquired the domain expertise
>>> required to use your DSL.
>>>
>> Yes, so if you believe me when I say that my DSL is orders of 
>> magnitude easier to use than
>> to use ANTLR to build and walk ASTs, then you must see my point: He's 
>> much more productive.
>> So I'm building my DSL (and other code) on top of ANTLR/lexer. I 
>> think there's an opportunity
>> for Terence to build a better and different tool in place of the 
>> ANTLR/parser - one that doesn't
>> require users to know formal language theory or picture ASTs in their 
>> heads.
>
>
> But what if the problem can only be specified in terms of formal 
> language theory? 

Then you'd have a different situation. But surely you're not saying that 
a language-to-language translator
can only be specified in terms of formal language theory, are you?

> There is something called "intelligence". If you don't have sufficient 
> intelligence, you *cannot* learn certain things. You want to give a 
> programmer a tool, so you can set him a problem he is INCAPABLE of 
> comprehending. I hope I never have to rely on (or even USE) code 
> written by a coder like that !!!

Well, virtually every coder in the world is like that. Only a handful of 
people in the world know the details of what their
compiler is doing as they use it every day. Same goes for the OS and the 
hardware. Same goes for  the car you drive
and just about everything other non-trivial machine you use: you use it 
without knowing much about how it works.
That's what makes the world go round :)

>>
>>>> I want to do. I *never* have to look at java byte code - I'm 
>>>> completely hidden from that
>>>> by the design of the compiler.
>>>>
>>>
>>> Not by the design of the compiler. But by how well tested it is. And 
>>> by how
>>> well documented Java (and javacc) is.
>>
>> No, I do think it's by the design of the compiler - by the design of 
>> compilers in general.
>> Compiler designers take it as a given that users need only know the 
>> syntax/semantics of the input
>> language. If Ter took it as a given that ANTLR4 users need only know 
>> the syntax/semantics
>> of the input language, he'd end up with a very different tool.
>
>
> That would appeal to a completely different set of users. He's written 
> a tool that does what he needs, and it appeals to users with similar 
> needs.

Just as the first assembler writers built tools that solved their own 
needs. Eventually, they were able to completely hide
the lower level detail, and attack problems at a whole higher level of 
thinking. Same for compiler writers.

>
>
> Maybe he will write your version of Antlr 4, but it will be a compiler 
> not a lexer/parser - a completely different tool!
>
>>
>>> Many Javacc users - who can and care
>>> to look under hood - have tripped over bugs-a-plenty in it yet, 
>>> "average"
>>> Javacc users don't discover those same bugs in the same tool even 
>>> when they
>>> write code that triggers it.
>>>
>>> Knowledge of Java and bytecode and how javacc works means "above 
>>> average"
>>> users can do more with Java and javacc than the average programmer can.
>>> Knowing more, lets you do more.
>>>
>>> ANTLR is similar and different. Different because it suffers in 
>>> comparison
>>> by being a less popular tool with less resources behind it. 
>>> Nevertheless,
>>> for someone with a deep knowledge of ANTLR's grammar language, it's
>>> limitations (e.g. no predicate hoisting and approximate-LLk in 2.x) 
>>> and the
>>> available documentation, there really is no need to look at the 
>>> output code.
>>>
>> Well, sure, it's mostly covered in the documentation. But that 
>> doesn't change the fact that it could be
>> made a lot easier (IMO). What makes Java great and C++ not-so-great 
>> is that Java programmers
>> never have to go look up the syntax for "pointer to function that 
>> returns an int pointer". Similarly,
>> a tool that didn't make me look up how to figure out how many 
>> patterns a "*" matched would make
>> the tool great.
>>
> And then the programmer needs a pointer to write to a bit of hardware, 
> and throws Java out in disgust because his "great" tool is useless for 
> the job at hand ...

Right. And probably gets fired if his boss finds out that he's trying to 
write a bit directly to hardware.
He thinks he needs to do that, and he thinks he's smart, but he's not. 
That's pretty rare these days to
have anyone feel that they need direct access to the hardware. We're 
many levels beyond that now.

>
>>> You develop your grammar, you develop your tests, you build the 
>>> whole lot
>>> and, the tests will alert you if you need to change anything. Unless 
>>> you
>>> trip over a bug in ANTLR of course...
>>>
>>> Now with ANTLR V3, not only can you look at the output code it if 
>>> you wish
>>> but, for the price of a little more knowledge (i.e. domain 
>>> expertise), you
>>> can change it!.
>>
>> I can change the bytecode generated by javac, too. If javac let me do 
>> that, it would be an indication
>> to the javac designer that his design is less-than-great.
>>
> Because javac is meant to convert java source accurately to java 
> bytecode. While Antlr is meant to (and does) accurately convert its 
> sourcecode into a lexer/parser/treewalker, it is also designed to let 
> you do things beyond the power of the tool. Both fit their design aims 
> - javac creates a java program, Antlr creates an extensible 
> lexer/parser/treewalker. Both are *good* tools, that doesn't 
> necessarily mean they are *great* tools.
>
> At the end of the day, I think your definition of "great" is badly 
> flawed - "a great tool lets a mediocre programmer do a decent job". A 
> great programmer could probably outperform that mediocre programmer 
> without that tool. 

Heh, it's been a while since I heard the "A great programmer can write 
better code than the compiler" line.
Used to hear that a lot 20 years ago, not much any more.

> To my mind, "a great tool lets a great programmer give of his best". 
> And a mediocre programmer would not be able to use that tool, because 
> he would not be able to understand it.
>
> There's a BIG difference between the RIGHT tool, and a GREAT tool. 
> Would you call a hammer a *great* tool when you have a pack of nails? 
> I don't know as I'd call a nailgun a great tool but it's certainly 
> greater than a hammer. And it requires a greater level of *maturity* 
> than a hammer to use safely.

The nailgun is greater than the hammer because it does the same job 
better *without* forcing the user to know the details
(how it works internally). If you had to constantly tweak the nailgun 
internals to get things done, it might still be "good" or
even "great", but not as good as it could be.

>
> The greater the tool, the greater the level of maturity required to be 
> able to use it safely. 

If you want to use that meaning for "great", that's fine. But I do 
disagree. If you had one space shuttle
that was completely safe and automated, that would be a "greater" tool 
than one that requires maturity, IMO.
Especially if you have millions of shuttle pilots, and not just a few.

> And the fewer the number of programmers who are capable of reaching 
> that level of maturity ... you go on about the "dangerous features" of 
> C++ as opposed to Java. A mature C++ programmer would NOT USE those 
> features if they weren't needed. Indeed, a mature C++ programmer would 
> probably use Java if those features weren't needed.

Haven't heard that one in a while either. Yes, yes, I know, it's not C++ 
that's dangerous, it's just those *other* guys
who are bad programmers and are misusing it. And we Java fans are just 
not manly enough to handle
MI and operator overloading.

>
>
> Too many programmers use features like kids eat candy - the more the 
> better! A great programmer likes a tool like a candy shop - but he has 
> the self control to only take what he needs - not the entire shop!

Yea, yea, heard it all before. It's all those *other*, less-intelligent 
programmers that are using operator overloading
to apply the "+" operator to the "Person" class. We smart programmers 
know that we *NEED* operator overloading
for our matrix multiplication. Bleh. :)

>
> Cheers,
> Wol