[antlr-interest] philosophy about translation

[Micheal J]

Hi Andy,

> I don't think I'm changing my position. I still disagree with that 
> quote. I think almost all the great tools
> are the ones that the majority of programmers to "get". In 
> fact, that's 
> part of what makes them great.

I don't think "popular" and "great" (when applied to tools) are synonyms.

Some "great" tools are accessible to the majority. Others are not.

> >Being a great tool for the job doesn't guarantee popularity. 
> Popularity 
> >is ultimately a measure of the tool's accessibility to average 
> >programmers (they are the majority). Great tools are often 
> beyond the 
> >ability of the average programmer. Certainly to build. And 
> often to use 
> >too.
> >  
> >
> I disagree. I view Java as being "great" and C++ not being 
> "great".

If productivity (not power) is the priority, Java/C# is likely to be a
better tool than C++ for those problems to which Java is applicable.

Otherwise C++ (or some other tool) is better.

> >C++ is a great tool (Java was written using it). Most Java 
> programmers
> >wouldn't be able to master it. Or the domain expertise 
> needed to build 
> >Java itself.
> >  
> >
> Well, I think Java is far better than C++,

I thought you might.

> and I think the 
> vast majority 
> of programmers who know both
> agree with me.

I make no such claim (I have no idea what the vast majority of competent
Java/C++ programmers think).

> Probably the main benefit is that it's easy to use for 
> "average programmers". That's also why
> ANTLR is better than the competition - because it's easier to use.

Not to my mind. Coco/R, JavaCC, SLK are equally easy to use (if one takes
the time to learn them).

ANTLR's "popularity" is due to a lot if things including: Ter's predicated
LL(k)/LL(*) technology, [somewhat] comprehensible code generation for
multiple target languages, grammar as documentation for all phases of
translation, it's PD or BSD license, funky codegen in V3 etc

> Yes, a few people want to add stuff back, but most do not. It's just 
> that the few are very vocal.
> The vast majority don't want MI, operator overloading, or built-in 
> AspectOrientedDesign.

The "vast majority" don't understand the value/utility of MI, mixins, or the
"why?" of AOD  etc.

> >>And that's why Java is popular and
> >>Smalltalk and 
> >>LISP are not.
> >>It's also why people prefer Java over C++.
> >>    
> >It is a easier tool to use. Less powerful. But easier.
> >  
> Right - so I hope there's nothing wrong with me pushing to make ANTLR 
> (or some successor)
> easier to use. A compiler is easier to use than compiling by 
> hand, but 
> also less powerful.
> I'm ok with that.

In the context of this thread, "compiling by hand" is not a tool.

How would you change in ANTLR to make it easier?

> >>Sometimes they do, but sometimes they don't. Compilers never require
> >>in-depth
> >>domain expertise.

> >Try feeding Java code or an ANTLR grammar to a C++ compiler. ;-)

> Of course you need to know the input syntax and symantics (a little). 
> That's different from
> knowing the details of what the tool is doing.

> That's like having to know the details about the 
> bytecode that
> javac creates. I don't have to read the manual for that stuff...I'd 
> rather have the tool not
> force me to know those details.

Quite often just getting "something that works" is all that is required.
Getting the best output from a compiler requires knowing more about what
goes on under the hood.

> >For your examples of [general purpose language] compiler and ANTLR 
> >[grammar language compiler], the domain expertise isn't 
> primarily about 
> >the internals of the tool. It's about the syntax, semantics 
> and idioms 
> >of the language recognized by the tool. Knowledge of the tool's 
> >internals can elevate those who have it above the "average" user who 
> >doesn't.
> >  
> >
> But required knowledge of the tool's internals limits the "average" 
> user's productivity.

A user is already limited if he/she don't understand how a tool works.
Whether or not that matters depends on what they are trying to accomplish.

> >Java's swan song is productivity (for those problems to 
> which to it can 
> >be applied). Not power as in flexibility, expressivity or 
> performance.
> >
> >I use Java/C# for the productivity benefits. If performance, 
> >flexibility or expressivity was *more* important in a particular 
> >project, there are better tools than Java/C# (e.g. C++, Occaml).
> >  
> >
> Right, so you're just like the rest of us. You've chosen to 
> limit your 
> own "power" by using Java rather
> than, say, assembly. So I'm sticking with my claim that
> "I think a tool can be great while being simple enough for most 
> programmers (e.g. Java)."
> and not buying your "Not without limiting it's power" reply.

It isn't "my" reply. The fact is:
- Java/C# is less powerful than assembler, C or C++ (you need them to build
java/c# in the first place).
- For some problems, Java/C# is more productive than assembler, C or C++.

> >Incidentally, your DSL is just a small part of your 
> particular language 
> >recognition toolkit.
> >
> It is??? How do you know that?

Relying on what I've learned about similar systems.
If it isn't I'd like to hear more about it.

> >He isn't using ANTLR directly (i.e. creating/maintaining ANTLR 
> >grammars) so, no surprise if he hasn't had to learn to use ANTLR.
> >  
> >
> Ah, but he is using ANTLR directly: he spends all day working 
> with the 
> Token streams produced by ANTLR,
> without having ever seen an ANTLR grammar. That's possible when using 
> ANTLR as a lexer,
> but that wouldn't be possible using its parser.

I disagree. He is working with code generated by ANTLR. He isn't using
ANTLR.

> >He _is_ using a DSL you created to encode source-to-source 
> >transformations. You just expressed the opinion that he has 
> aquired the 
> >domain expertise required to use your DSL.
> >  
> >
> Yes, so if you believe me when I say that my DSL is orders of 
> magnitude 
> easier to use than
> to use ANTLR to build and walk ASTs, then you must see my point: He's 
> much more productive.

But he is also limited to what your DSL allows - cf. "not without limiting
it's power". 

If he used ANTLR directly (like you did). He can do more. Your DSL (like
Java/C#) favours productivity over power/flexibility.

> So I'm building my DSL (and other code) on top of 
> ANTLR/lexer. I think 
> there's an opportunity
> for Terence to build a better and different tool in place of the 
> ANTLR/parser - one that doesn't
> require users to know formal language theory or picture ASTs in their 
> heads.

Personally, I can't see how that is possible.

> >Not by the design of the compiler. But by how well tested it 
> is. And by 
> >how well documented Java (and javacc) is.
> >
> No, I do think it's by the design of the compiler - by the design of 
> compilers in general.

ANTLR *is* a compiler.

> Compiler designers take it as a given that users need only know the 
> syntax/semantics of the input
> language. If Ter took it as a given that ANTLR4 users need 
> only know the 
> syntax/semantics
> of the input language, he'd end up with a very different tool.

When using ANTLR, that is all one needs to know. By knowing even more, it is
possible to do even more than the "average" ANTLR user.

> >Now with ANTLR V3, not only can you look at the output code 
> it if you 
> >wish but, for the price of a little more knowledge (i.e. domain 
> >expertise), you can change it!.
> >  
> >
> I can change the bytecode generated by javac, too. If javac let me do 
> that, it would be an indication
> to the javac designer that his design is less-than-great.

Or that "the [above average] programmer knows best" to paraphrase the C/C++
motto.

A compiler designer can't determine the best code to generate for every
possible situation in advance. This feature makes the tool more useful - for
those who care to acquire the knowledge required to use it effectively. It
empowers knowledgeable users to tailor the output for any given situation.


Micheal