[antlr-interest] philosophy about translation

[Andy Tripp]

Micheal J wrote:

>Andy,
>
>You seem to be changing your position on the issue of whether (as Anthony
>put it):
>
>"If it really *is* great, then the chances are the majority of programmers
>*can't* 'easily "get it" '."
>
>I'm leaning towards agreeing with Anthony as I said.
>  
>
I don't think I'm changing my position. I still disagree with that 
quote. I think almost all the great tools
are the ones that the majority of programmers to "get". In fact, that's 
part of what makes them great.

>>I view Java's decision to do a "limited implementation" by avoiding 
>>things like multiple inheritance
>>as exactly what made it successful.
>>    
>>
>
>Being a great tool for the job doesn't guarantee popularity. Popularity is
>ultimately a measure of the tool's accessibility to average programmers
>(they are the majority). Great tools are often beyond the ability of the
>average programmer. Certainly to build. And often to use too.
>  
>
I disagree. I view Java as being "great" and C++ not being "great". And 
the main thing that makes
Java great is that all the features that made it inaccessible to average 
programmers were removed.
And, it's no coincidence that these same features are what always got 
the supposedly "great programmers"
into trouble: pointers, multiple inheritence, operator overloading, etc.

>C++ is a great tool (Java was written using it). Most Java programmers
>wouldn't be able to master it. Or the domain expertise needed to build Java
>itself.
>  
>
Well, I think Java is far better than C++, and I think the vast majority 
of programmers who know both
agree with me. Probably the main benefit is that it's easy to use for 
"average programmers". That's also why
ANTLR is better than the competition - because it's easier to use.

>  
>
>>By avoiding being 
>>"completely pure", 
>>Java is accessible
>>to average programmers.
>>    
>>
>
>My point exactly (not sure about the "pure OO" label though). 
>
>Incidentally "above average" Java programmers understand the value of the
>missing features and are forever trying to add them back. As I suggested,
>AspectOrientedDesign in Java can be viewed as attempts to hack some of them
>back into Java.
>  
>
Yes, a few people want to add stuff back, but most do not. It's just 
that the few are very vocal.
The vast majority don't want MI, operator overloading, or built-in 
AspectOrientedDesign.
And of course, those who want them think they're "above average", but 
then so does almost
everyone.

>  
>
>>And that's why Java is popular and 
>>Smalltalk and 
>>LISP are not.
>>It's also why people prefer Java over C++.
>>    
>>
>
>It is a easier tool to use. Less powerful. But easier.
>  
>
Right - so I hope there's nothing wrong with me pushing to make ANTLR 
(or some successor)
easier to use. A compiler is easier to use than compiling by hand, but 
also less powerful.
I'm ok with that.

>  
>
>>>So, I'm tending to agree with Anthony here. Great tools 
>>>      
>>>
>>often require 
>>    
>>
>>>in-depth domain expertise that the majority simply don't have.
>>> 
>>>
>>>      
>>>
>>Sometimes they do, but sometimes they don't. Compilers never require 
>>in-depth
>>domain expertise.
>>    
>>
>
>Try feeding Java code or an ANTLR grammar to a C++ compiler. ;-)
>  
>
Of course you need to know the input syntax and symantics (a little). 
That's different from
knowing the details of what the tool is doing. When talking to Ter, at 
one point I said
"and how do I know how many times that pattern was matched?" and he said 
"just check
n < 4". How was I supposed to know that ANTLR generated some code that 
included
a variable called n? That's like having to know the details about the 
bytecode that
javac creates. I don't have to read the manual for that stuff...I'd 
rather have the tool not
force me to know those details.

>  
>
>>I know almost nothing about byte-code 
>>generation, yet 
>>I use javac
>>every minute or two. I think the world would benefit from an 
>>ANTLR tool 
>>that was like that.
>>    
>>
>
>For your examples of [general purpose language] compiler and ANTLR [grammar
>language compiler], the domain expertise isn't primarily about the internals
>of the tool. It's about the syntax, semantics and idioms of the language
>recognized by the tool. Knowledge of the tool's internals can elevate those
>who have it above the "average" user who doesn't.
>  
>
But required knowledge of the tool's internals limits the "average" 
user's productivity.
If I had done AST-based translation, I'd be spending way to much time 
worrying about the details
of the AST, rather than the syntax and semantics of the two languages. I 
demand to spend 95% of
my time on *what* to do, rather than *how* to do it. With ASTs, I found 
myself spending
95% of my time on *how*.

>  
>
>>>>I think a tool can be great while being simple enough for most
>>>>programmers (e.g. Java).
>>>>   
>>>>
>>>>        
>>>>
>>>Not without limiting it's power.
>>> 
>>>
>>>      
>>>
>>Yes, just as Java's power is "limited" by not supporting MI, 
>>pointers, etc. I love to have my power "limited" by not 
>>giving me lots of rope to hang 
>>myself with.
>>And so do most people, judging by the popularity of Java over C++ and 
>>every high-level
>>language over assembly.
>>    
>>
>
>Java's swan song is productivity (for those problems to which to it can be
>applied). Not power as in flexibility, expressivity or performance.
>
>I use Java/C# for the productivity benefits. If performance, flexibility or
>expressivity was *more* important in a particular project, there are better
>tools than Java/C# (e.g. C++, Occaml).
>  
>
Right, so you're just like the rest of us. You've chosen to limit your 
own "power" by using Java rather
than, say, assembly. So I'm sticking with my claim that
"I think a tool can be great while being simple enough for most 
programmers (e.g. Java)."
and not buying your "Not without limiting it's power" reply.

>
>>>      
>>>
>>Sure he could. Joe average could easily write:
>>a + b --> a.add(b)
>>and have his tool do the rest (and maybe warn him about cases 
>>that might 
>>match that he hadn't
>>thought about).
>>    
>>
>
>Not without understanding the syntax and semantics of the DSL you created.
>Not without understanding just what that input string instructs your DSL's
>"compiler" to do.
>  
>
There's little more to my DSL than knowing that "-->" is a separator, 
"v" matches a
single constant or variable tokens, and "x" matches zero or more tokens. 
I'd say it's several
orders of magnitude simpler than ANTLR. I'm not comparing it to ANTLR, 
just saying that
I do believe that I do believe that a does what ANTLR does, but is far 
simpler (like an order
of magnitude or two simpler).

>Incidentally, your DSL is just a small part of your particular language
>recognition toolkit. 
>
It is??? How do you know that?

>My comment refers to someone building the whole
>toolkit. Examples abound of tools that offer multiple DSLs to tackle the
>various phases/modules of a language processing toolkit (e.g. Cocktail,
>Stratego).
>  
>
I'm referring to my "whole toolkit" too, so I think we're talking about 
the same thing.

>  
>
>>I've had a programmer working with me for a few months now, 
>>and he's had 
>>no trouble writing
>>translation rules without every learning ANTLR grammar or knowing 
>>anything about language
>>recognition tools.
>>    
>>
>
>He isn't using ANTLR directly (i.e. creating/maintaining ANTLR grammars) so,
>no surprise if he hasn't had to learn to use ANTLR. 
>  
>
Ah, but he is using ANTLR directly: he spends all day working with the 
Token streams produced by ANTLR,
without having ever seen an ANTLR grammar. That's possible when using 
ANTLR as a lexer,
but that wouldn't be possible using its parser.

>He isn't developing a language translation toolkit (you've done that
>already) so, no surprise if he he has no grounding in formal language
>theory.
>
>He _is_ using a DSL you created to encode source-to-source transformations.
>You just expressed the opinion that he has aquired the domain expertise
>required to use your DSL.
>  
>
Yes, so if you believe me when I say that my DSL is orders of magnitude 
easier to use than
to use ANTLR to build and walk ASTs, then you must see my point: He's 
much more productive.
So I'm building my DSL (and other code) on top of ANTLR/lexer. I think 
there's an opportunity
for Terence to build a better and different tool in place of the 
ANTLR/parser - one that doesn't
require users to know formal language theory or picture ASTs in their 
heads.

>>I want to do. I *never* have to look at java byte code - I'm 
>>completely 
>>hidden from that
>>by the design of the compiler.
>>    
>>
>
>Not by the design of the compiler. But by how well tested it is. And by how
>well documented Java (and javacc) is. 
>
No, I do think it's by the design of the compiler - by the design of 
compilers in general.
Compiler designers take it as a given that users need only know the 
syntax/semantics of the input
language. If Ter took it as a given that ANTLR4 users need only know the 
syntax/semantics
of the input language, he'd end up with a very different tool.

>Many Javacc users - who can and care
>to look under hood - have tripped over bugs-a-plenty in it yet, "average"
>Javacc users don't discover those same bugs in the same tool even when they
>write code that triggers it.
>
>Knowledge of Java and bytecode and how javacc works means "above average"
>users can do more with Java and javacc than the average programmer can. 
>
>Knowing more, lets you do more.
>
>ANTLR is similar and different. Different because it suffers in comparison
>by being a less popular tool with less resources behind it. Nevertheless,
>for someone with a deep knowledge of ANTLR's grammar language, it's
>limitations (e.g. no predicate hoisting and approximate-LLk in 2.x) and the
>available documentation, there really is no need to look at the output code.
>  
>
Well, sure, it's mostly covered in the documentation. But that doesn't 
change the fact that it could be
made a lot easier (IMO). What makes Java great and C++ not-so-great is 
that Java programmers
never have to go look up the syntax for "pointer to function that 
returns an int pointer". Similarly,
a tool that didn't make me look up how to figure out how many patterns a 
"*" matched would make
the tool great.

>You develop your grammar, you develop your tests, you build the whole lot
>and, the tests will alert you if you need to change anything. Unless you
>trip over a bug in ANTLR of course...
>
>Now with ANTLR V3, not only can you look at the output code it if you wish
>but, for the price of a little more knowledge (i.e. domain expertise), you
>can change it!. 
>  
>
I can change the bytecode generated by javac, too. If javac let me do 
that, it would be an indication
to the javac designer that his design is less-than-great.

>The "average" ANTLR user has no need to change the code and would never do
>so but, others will.
>
>
>Micheal
>  
>
OK,
Andy

>
>  
>