[antlr-interest] philosophy about translation

[Andy Tripp]

>>
>> Every time I've heard anyone say they NEED MI, operator overloading, 
>> or Aspects, I've thought
>> that they really don't need them, they're just not good enough 
>> programmers to see a clean way
>> to do without them. I've been programming for 25 years without ever 
>> NEEDING any of these,
>> and so have most other programmers.
>>
> I programmed for many years without needing pointers. FORTRAN didn't 
> have them, so I tended not to use them in C. I notice you said *MOST* 
> other programmers. You're jumping to conclusions - because "all the 
> programmers I know are too stupid to live without these features" you 
> conclude that these features "mostly aren't needed". So you'd like to 
> see them removed from any tool you use ... :-(

I conclude that these features "mostly aren't needed" (really more like 
"almost never" or even "never") from personal
experience, not from something you or I said.

>>>
>>>>
>>> So - you're quite happy to see Antlr crippled to suit you, ignoring 
>>> the NEEDS of those who need its power!
>>
>>
>> Don't worry, I can't cripple ANTLR.  But yes, I'd be happy to see 
>> ANTLR4 become a "crippled" version,
>> in the same sense that Java is a "crippled" version of C++.
>>
> And as I said, you're quite happy to ruin the tool for all those 
> people who need that power ...

Yes, if you consider Java to be "ruining" C++. New tool without all the 
warts. Those
who "need the power" of the warts can keep using the old tool.

>
>>>>>
>>>>>
>>>> But required knowledge of the tool's internals limits the "average" 
>>>> user's productivity.
>>>> If I had done AST-based translation, I'd be spending way to much 
>>>> time worrying about the details
>>>> of the AST, rather than the syntax and semantics of the two 
>>>> languages. I demand to spend 95% of
>>>> my time on *what* to do, rather than *how* to do it. With ASTs, I 
>>>> found myself spending
>>>> 95% of my time on *how*.
>>>>
>>> In other words, as you said earlier, ASTs are the wrong tool for 
>>> you. So you seem happy to delete ASTs from Antlr because *you* don't 
>>> need them, irregardless of what other people *NEED*.
>>
>>
>> Yes, just as I railed against operator overloading and MI in C++, and 
>> found bliss in Java.
>> I don't much care that other people think they *NEED* MI. They can 
>> always stick with C++.
>>
> Or you could find some other language instead of Java? You'd much 
> rather inconvenience all the other Java programmers by removing 
> features they need, rather than convenience yourself by finding 
> something that suits you better?

No, you're messing up the metaphor. I'm proposing that the next version 
of ANTLR (or maybe it's a completely new tool)
be to today's ANTLR as Java is to C++: a new and improved version that's 
really better, in part because it takes programming
to a higher level.

>>>>>
>>
>>> Let's ask a question ... how can a tool be "great" if it *relies* on 
>>> other tools even for its existence?
>>
>>
>> Sure, of course! Virtually all software relies on other tools 
>> (compilers, operating systems, etc) for their existence.
>>
>>>
>>> A C compiler can compile itself. Can a Java system build itself? How 
>>> much of Java is actually written in Java? (Oh and I'm including the 
>>> supporting libraries here!)
>>
>>
>> Almost all of Java is written in Java. The only part that's not is 
>> the lowest layer, which is OS and hardware specific.
>>
> Assembler is written in assembler, pretty much without exception. C is 
> written in C, pretty much without exception.
>
> The reason for demanding that a great tool be written in itself is 
> exemplified by my databasic example above - the primitives for the 
> version I initially used were written in machine code, and followed 
> the spec. The C rewrite introduced loads of subtle bugs, because the 
> characteristics of the C environment were different. How much of Java 
> DEPENDS on the lowest layer? How much knowledge do you need to 
> understand that layer?

Heheh...didn't know that most of Java was written in Java, did you? ;)
Yes, all of Java DEPENDS on C.  All of C DEPENDS on object code. All of 
object code DEPENDS on
the operating system. All of the operating system DEPENDS on the hardware.

>>>
>>> Antlr v3 is due to be rewritten in Antlr v3. To my mind, that's a 
>>> "necessary but not sufficient" condition for greatness.
>>
>>
>> Wow. Only software that's written in itself can be "great"? Are you 
>> really saying that?
>>
> Yes I am. Because otherwise it requires you to understand much more 
> than you want about the execution environment. Note my comments about 
> subtle bugs. Because it requires the system guys to be *experts* in 
> two completely different environments!
>
> I note that you think a "great tool" is one that means you *don't* 
> *need* to be an expert. By your logic, Java therefore has to be a crap 
> tool for Java system programmers ...

No, I don't mean "...*dont need* to be an expert", but rather "dont need 
to know about the layers below". Java is great
because it doesn't require any knowledge about memory management, for 
example. The tool should let you
only be an expert *at one level of abstraction*. I can program in Java 
without knowing (almost) any details about
any of the underlying layers.

>
> Which is why, in the system I want to build with Antlr, it's very 
> important to me that AS MUCH AS POSSIBLE of the system is built using 
> DATABASIC, because that's what the programmers using it will have 
> experience in. *THAT* is why a great tool must be built using itself.

I think maybe you're only picturing cases where the tool developer is 
the same person as the tool user.
I don't care at all whether ANTLR (Or Java, or anything else I use) is 
built with itself. I'm sure Terence cares,
but he's the exception.

>
>>
>> Well, virtually every coder in the world is like that. Only a handful 
>> of people in the world know the details of what their
>> compiler is doing as they use it every day. Same goes for the OS and 
>> the hardware. Same goes for  the car you drive
>> and just about everything other non-trivial machine you use: you use 
>> it without knowing much about how it works.
>> That's what makes the world go round :)
>>
> You miss the point. Let's take the car example. There's a big 
> difference between what I wrote and you responded to. As far as I can 
> see, you're saying that if a guy has a driving licence, you're quite 
> happy to get him to repair your car brakes ... WHAT...!!!

No, I'm just saying that what makes the car useful is that you don't 
have to understand it to use it. You're
hidden from the layers below.

>
> You do not give a problem to someone to solve, if they do not have the 
> mental capacity to understand the problem. You do not ask a guy to 
> repair the brakes on your car if he has no experience of being a 
> mechanic, even if he does have a driving licence (and there's a lot of 
> people I would hate to have service my car, even if they do have a 
> mechanics qualification, because they learnt to follow the diagrams in 
> the book and didn't learn to work through what was *actually* 
> *happening*!).

I agree.

>>>>
>>> And then the programmer needs a pointer to write to a bit of 
>>> hardware, and throws Java out in disgust because his "great" tool is 
>>> useless for the job at hand ...
>>
>>
>> Right. And probably gets fired if his boss finds out that he's trying 
>> to write a bit directly to hardware.
>> He thinks he needs to do that, and he thinks he's smart, but he's 
>> not. That's pretty rare these days to
>> have anyone feel that they need direct access to the hardware. We're 
>> many levels beyond that now.
>>
> Given the ability of many bosses, I bet the boss gave him the job, and 
> told him to use the wrong tool ...

Maybe. From my experience, almost always when someone says "I need to 
set individual bits by hand,
so Java isn't a good fit", it turns out that in fact he does not need to 
set individual bits by hand, and Java will
work just fine. He just thinks he needs to bit-twiddle because that's 
how he's used to doing things, and he
wouldn't even think to just use an array of booleans because he can't 
make that mental leap.

>
>>>
>>> Because javac is meant to convert java source accurately to java 
>>> bytecode. While Antlr is meant to (and does) accurately convert its 
>>> sourcecode into a lexer/parser/treewalker, it is also designed to 
>>> let do things beyond the power of the tool. Both fit their design 
>>> aims  - javac creates a java program, Antlr creates an extensible 
>>> lexer/parser/treewalker. Both are *good* tools, that doesn't 
>>> necessarily mean they are *great* tools.
>>>
>>> At the end of the day, I think your definition of "great" is badly 
>>> flawed - "a great tool lets a mediocre programmer do a decent job". 
>>> A great programmer could probably outperform that mediocre 
>>> programmer without that tool.
>>
>>
>> Heh, it's been a while since I heard the "A great programmer can 
>> write better code than the compiler" line.
>> Used to hear that a lot 20 years ago, not much any more.
>
>
> I gather the statistics still hold true. The best programmers still 
> outperform the average ones by about 1000% ... that's a MASSIVE 
> difference.

Yea, that's about right. But the best compilers now outperform the best 
programmers at compiling. We're
comparing people to tools above, not people to people.

>
>>
>>>
>>> The greater the tool, the greater the level of maturity required to 
>>> be able to use it safely.
>>
>>
>> If you want to use that meaning for "great", that's fine. But I do 
>> disagree. If you had one space shuttle
>> that was completely safe and automated, that would be a "greater" 
>> tool than one that requires maturity, IMO.
>> Especially if you have millions of shuttle pilots, and not just a few.
>
>
> You've obviously not read Dick Feynmann then ... :-) even *I* could 
> fly the space shuttle :-)

"Surely You're Joking!" ;) Maybe you could point me to where he talks 
about this issue,
because I don't remember it.

>
>>
> Fine. Since Java is such a great language, let's see you write the 
> BIOS for all new computers in it ... what? you mean all new computers 
> will be bricks if you do? What a crap tool ...

Just because a car can't fly doesn't mean it's not a great tool. ;)

>
> Oh - and as for "we smart programmers", you do realise you've just 
> disqualified yourself as a member of that set? You used the word "we"! 
> If the "other" guy respects you, he'll do as you say. I've done some 
> crazy things in my time, like using a goto in C :-) But when other 
> programmers have said "you shouldn't do that" I've just said "well, 
> you do better". Usually, they've looked at the problem to be solved, 
> and backed down. If you're a smart programmer, you'll show the other 
> guy how to do it better. If you can't, he's smarter than you thought!

I'll take that challenge...show me a piece of code that really needs a 
goto, and I'll remove it (assuming you let me rewrite
the whole darn thing if I need to).

>
> "Perfection is achieved, not when everything that is necessary has 
> been added, but when everything that is unnecessary has been taken away".

And so it is with C++ and Java.

>
>
> I *know* I'm an expert in DATABASIC. I would *like* to *think* I'm a 
> guru, but until somebody like Jim, or Rob, or Monty, tells me I am 
> (and I probably wouldn't believe Monty), then I'm not! And even then, 
> I wouldn't believe them unless other people I trusted agreed with them.
>
> Ask Loring what unnecessary stuff can be removed from Antlr. Ask Ter. 
> Ask Jim. Ask Monty. Ask the people *YOU* *RESPECT* *AS* *EXPERTS* (or 
> aren't there any such on this list? :-)

Ask Bjourne what unnecessary stuff can be removed from C++. Now ask 
Gosling. Often, the experts are the
exact wrong people to ask.

>
> I think you'll find everything in Antlr is there because it's needed. 
> Like ASTs for example - I think they happen to be a perfect solution 
> to my problem :-)
>
> I know Antlr doesn't do what you want. So stop trying to drive screws 
> with a hammer, and go find yourself a screwdriver :-) Don't try and 
> ban hammers :-)

You do understand that I'm not trying to "disable" or "take away 
features" from ANTLR, right?
I think it's perfectly reasonable for someone to post his crazy dreams 
for an "electric hammer" on
a "Joe's hammer" forum. If you don't want to read it, then don't.

>
> (Incidentally, one of the features I found Antlr lacked - or rather 
> seemed incredibly complex to implement - in v2 was a stream rewriter - 
> I wanted a state engine to flip token types - certain types may or may 
> not follow other types and I could do it a lot easier in a stream 
> rewriter than by tracking state in the lexer. Maybe that would be a 
> useful tool for you, maybe not.)

Probably not, as I can't imagine how it could be easier than looping 
through the List of tokens "by hand".

Andy

>
> Cheers,
> Wol