[antlr-interest] philosophy about translation

[Anthony W. Youngman]

In message <4548F943.7070906 at jazillian.com>, Andy Tripp 
<antlr at jazillian.com> writes
>Micheal J wrote:
>
>>Andy,
>>
>>You seem to be changing your position on the issue of whether (as Anthony
>>put it):
>>
>>"If it really *is* great, then the chances are the majority of programmers
>>*can't* 'easily "get it" '."
>>
>>I'm leaning towards agreeing with Anthony as I said.
>>
>I don't think I'm changing my position. I still disagree with that 
>quote. I think almost all the great tools
>are the ones that the majority of programmers to "get". In fact, that's 
>part of what makes them great.

I wouldn't describe "popular" as "great".
>
>>>I view Java's decision to do a "limited implementation" by avoiding 
>>>things like multiple inheritance
>>>as exactly what made it successful.
>>>
>>
>>Being a great tool for the job doesn't guarantee popularity. Popularity is
>>ultimately a measure of the tool's accessibility to average programmers
>>(they are the majority). Great tools are often beyond the ability of the
>>average programmer. Certainly to build. And often to use too.
>>
>I disagree. I view Java as being "great" and C++ not being "great". And 
>the main thing that makes
>Java great is that all the features that made it inaccessible to 
>average programmers were removed.
>And, it's no coincidence that these same features are what always got 
>the supposedly "great programmers"
>into trouble: pointers, multiple inheritence, operator overloading, etc.
>
>>C++ is a great tool (Java was written using it). Most Java programmers
>>wouldn't be able to master it. Or the domain expertise needed to build Java
>>itself.
>>
>Well, I think Java is far better than C++, and I think the vast 
>majority of programmers who know both
>agree with me. Probably the main benefit is that it's easy to use for 
>"average programmers". That's also why
>ANTLR is better than the competition - because it's easier to use.
>
You've just said Java doesn't have pointers. So how do you access a 
hardware register in Java? How do you access a screen buffer?
>>
>>>By avoiding being "completely pure", Java is accessible
>>>to average programmers.
>>>
>>
>>My point exactly (not sure about the "pure OO" label though).
>>Incidentally "above average" Java programmers understand the value of the
>>missing features and are forever trying to add them back. As I suggested,
>>AspectOrientedDesign in Java can be viewed as attempts to hack some of them
>>back into Java.
>>
>Yes, a few people want to add stuff back, but most do not. It's just 
>that the few are very vocal.
>The vast majority don't want MI, operator overloading, or built-in 
>AspectOrientedDesign.
>And of course, those who want them think they're "above average", but 
>then so does almost
>everyone.
>
What about those who don't WANT those things, but NEED them?
>>
>>>And that's why Java is popular and Smalltalk and LISP are not.
>>>It's also why people prefer Java over C++.
>>>
>>
>>It is a easier tool to use. Less powerful. But easier.
>>
>Right - so I hope there's nothing wrong with me pushing to make ANTLR 
>(or some successor)
>easier to use. A compiler is easier to use than compiling by hand, but 
>also less powerful.
>I'm ok with that.
>
So - you're quite happy to see Antlr crippled to suit you, ignoring the 
NEEDS of those who need its power!
>>
>>
>But required knowledge of the tool's internals limits the "average" 
>user's productivity.
>If I had done AST-based translation, I'd be spending way to much time 
>worrying about the details
>of the AST, rather than the syntax and semantics of the two languages. 
>I demand to spend 95% of
>my time on *what* to do, rather than *how* to do it. With ASTs, I found 
>myself spending
>95% of my time on *how*.
>
In other words, as you said earlier, ASTs are the wrong tool for you. So 
you seem happy to delete ASTs from Antlr because *you* don't need them, 
irregardless of what other people *NEED*.
>>
>>>>>I think a tool can be great while being simple enough for most
>>>>>programmers (e.g. Java).
>>>>>
>>>>>
>>>>Not without limiting it's power.
>>>>
>>>>
>>>Yes, just as Java's power is "limited" by not supporting MI, 
>>>pointers, etc. I love to have my power "limited" by not giving me 
>>>lots of rope to hang myself with.
>>>And so do most people, judging by the popularity of Java over C++ and 
>>>every high-level
>>>language over assembly.
>>>
>>
>>Java's swan song is productivity (for those problems to which to it can be
>>applied). Not power as in flexibility, expressivity or performance.
>>
>>I use Java/C# for the productivity benefits. If performance, flexibility or
>>expressivity was *more* important in a particular project, there are better
>>tools than Java/C# (e.g. C++, Occaml).
>>
>Right, so you're just like the rest of us. You've chosen to limit your 
>own "power" by using Java rather
>than, say, assembly. So I'm sticking with my claim that
>"I think a tool can be great while being simple enough for most 
>programmers (e.g. Java)."
>and not buying your "Not without limiting it's power" reply.
>
To my mind, that's not a "great" tool. As I said above, "popular", 
maybe.

Let's ask a question ... how can a tool be "great" if it *relies* on 
other tools even for its existence?

A C compiler can compile itself. Can a Java system build itself? How 
much of Java is actually written in Java? (Oh and I'm including the 
supporting libraries here!)

Antlr v3 is due to be rewritten in Antlr v3. To my mind, that's a 
"necessary but not sufficient" condition for greatness.
>>
>>
>>He _is_ using a DSL you created to encode source-to-source transformations.
>>You just expressed the opinion that he has aquired the domain expertise
>>required to use your DSL.
>>
>Yes, so if you believe me when I say that my DSL is orders of magnitude 
>easier to use than
>to use ANTLR to build and walk ASTs, then you must see my point: He's 
>much more productive.
>So I'm building my DSL (and other code) on top of ANTLR/lexer. I think 
>there's an opportunity
>for Terence to build a better and different tool in place of the 
>ANTLR/parser - one that doesn't
>require users to know formal language theory or picture ASTs in their 
>heads.

But what if the problem can only be specified in terms of formal 
language theory? There is something called "intelligence". If you don't 
have sufficient intelligence, you *cannot* learn certain things. You 
want to give a programmer a tool, so you can set him a problem he is 
INCAPABLE of comprehending. I hope I never have to rely on (or even USE) 
code written by a coder like that !!!
>
>>>I want to do. I *never* have to look at java byte code - I'm 
>>>completely hidden from that
>>>by the design of the compiler.
>>>
>>
>>Not by the design of the compiler. But by how well tested it is. And by how
>>well documented Java (and javacc) is.
>No, I do think it's by the design of the compiler - by the design of 
>compilers in general.
>Compiler designers take it as a given that users need only know the 
>syntax/semantics of the input
>language. If Ter took it as a given that ANTLR4 users need only know 
>the syntax/semantics
>of the input language, he'd end up with a very different tool.

That would appeal to a completely different set of users. He's written a 
tool that does what he needs, and it appeals to users with similar 
needs.

Maybe he will write your version of Antlr 4, but it will be a compiler 
not a lexer/parser - a completely different tool!
>
>>Many Javacc users - who can and care
>>to look under hood - have tripped over bugs-a-plenty in it yet, "average"
>>Javacc users don't discover those same bugs in the same tool even when they
>>write code that triggers it.
>>
>>Knowledge of Java and bytecode and how javacc works means "above average"
>>users can do more with Java and javacc than the average programmer 
>>can.
>>Knowing more, lets you do more.
>>
>>ANTLR is similar and different. Different because it suffers in comparison
>>by being a less popular tool with less resources behind it. Nevertheless,
>>for someone with a deep knowledge of ANTLR's grammar language, it's
>>limitations (e.g. no predicate hoisting and approximate-LLk in 2.x) and the
>>available documentation, there really is no need to look at the output code.
>>
>Well, sure, it's mostly covered in the documentation. But that doesn't 
>change the fact that it could be
>made a lot easier (IMO). What makes Java great and C++ not-so-great is 
>that Java programmers
>never have to go look up the syntax for "pointer to function that 
>returns an int pointer". Similarly,
>a tool that didn't make me look up how to figure out how many patterns 
>a "*" matched would make
>the tool great.
>
And then the programmer needs a pointer to write to a bit of hardware, 
and throws Java out in disgust because his "great" tool is useless for 
the job at hand ...

>>You develop your grammar, you develop your tests, you build the whole lot
>>and, the tests will alert you if you need to change anything. Unless you
>>trip over a bug in ANTLR of course...
>>
>>Now with ANTLR V3, not only can you look at the output code it if you wish
>>but, for the price of a little more knowledge (i.e. domain expertise), you
>>can change it!.
>I can change the bytecode generated by javac, too. If javac let me do 
>that, it would be an indication
>to the javac designer that his design is less-than-great.
>
Because javac is meant to convert java source accurately to java 
bytecode. While Antlr is meant to (and does) accurately convert its 
sourcecode into a lexer/parser/treewalker, it is also designed to let 
you do things beyond the power of the tool. Both fit their design aims - 
javac creates a java program, Antlr creates an extensible 
lexer/parser/treewalker. Both are *good* tools, that doesn't necessarily 
mean they are *great* tools.

At the end of the day, I think your definition of "great" is badly 
flawed - "a great tool lets a mediocre programmer do a decent job". A 
great programmer could probably outperform that mediocre programmer 
without that tool. To my mind, "a great tool lets a great programmer 
give of his best". And a mediocre programmer would not be able to use 
that tool, because he would not be able to understand it.

There's a BIG difference between the RIGHT tool, and a GREAT tool. Would 
you call a hammer a *great* tool when you have a pack of nails? I don't 
know as I'd call a nailgun a great tool but it's certainly greater than 
a hammer. And it requires a greater level of *maturity* than a hammer to 
use safely.

The greater the tool, the greater the level of maturity required to be 
able to use it safely. And the fewer the number of programmers who are 
capable of reaching that level of maturity ... you go on about the 
"dangerous features" of C++ as opposed to Java. A mature C++ programmer 
would NOT USE those features if they weren't needed. Indeed, a mature 
C++ programmer would probably use Java if those features weren't needed.

Too many programmers use features like kids eat candy - the more the 
better! A great programmer likes a tool like a candy shop - but he has 
the self control to only take what he needs - not the entire shop!

Cheers,
Wol
-- 
Anthony W. Youngman - anthony at thewolery.demon.co.uk