[antlr-interest] Why no links to ANTLR 3.0 on www.antlr.org??
Geir Ove Skjaervik
geiroves at online.no
Thu Jun 9 11:42:05 PDT 2005
Hello Terence,
(got your name right this time I hope !)
<<the problem you have though is a general "how do I build an
interpreter" problem, which has nothing to do with ANTLR really. ANTLR
is just the structure recognizer. You need to learn about symbol
tables, exec environments, and a bunch of other stuff. Perhaps my
grad course notes can help you out there.>>
Well, I do understand symbol tables et. Al: I designed my own language
& wrote the Parser and Stack machine for it 10 years ago in 6 weeks, and
the current ANTLR version is also running fine. However, I had the
impression that ANTLR was indeed built to build interpreters: Why else
all these Tree Walking examples that adds numbers?
Anway, it is quite possible to build Interpreters with ANTLR, and even
though I see places where ANTLR code can be improved, the current
version I have runs 5 times faster (in Java) than the C++ Stack Machine
version that was written 10 years ago on the **same** computer!
The problems is that after a while technical terms will be forgotten:
Literals, Tokens, String literals etc. It would be nice if ANTLR did a
short **practical** recap: I do know the current docs explains Tokens
and Virtual Tokens, and today when I read it, I think I understand it:
However, I did not the first times I read it!
I do really look forward to your book(s) on ANTLR: If I have the time, I
will be happy to comment on the contents.
Hope my feedback has not been too negatively biased: I was not meant to
be; but I admit to some frustration over getting stuck in the docs :-)
Finally a resounding THANKS to you for bringing this great tool to the
Parsing Community :-) Hope that you don't forget that ANTLR can indeed
be used to interpret code; That is at least my main goal using ANTLR!
I will also write a debugger for the same Tree Walker Interpreter, any
hints there?
Regards
Geir Ove
-----Original Message-----
From: antlr-interest-bounces at antlr.org
[mailto:antlr-interest-bounces at antlr.org] On Behalf Of Terence Parr
Sent: 9. juni 2005 19:36
To: ANTLR Interest
Subject: Re: [antlr-interest] Why no links to ANTLR 3.0 on
www.antlr.org??
On Jun 9, 2005, at 5:12 AM, Geir Ove Skjaervik wrote:
> Hello Terrnce,
Hi !
> The way you respond to mr. Blomberg tells me that at your level of
> knowlegde of Parsing and Parser technology you do not any longer
> comprehend the problems beginners are struggling with: You ask
> questions
> the novice / beginner does NOT know how to answer because (he / she /
> we) are trying to grasp the concept.
Well, I do have to teach students this stuff on a regular
basis...seems I succeed sometimes...
> Please do take this as constructive feedback as better
> documentation is
> indeed needed to get the beginners going.
I appreciate all feedback! I want my doc to be as helpful as
possible. I plan on an O'Reilly style ANTLR book first then a
"language translation" book next.
> I have completed my first small Programming Language in ANTLR that
> exeuctes by Tree Walking,
Good for you!
> and I still do not fully understand what you
> ask Mr. Blomberg... Maybe again it's me...
>
> E.g. You asked:
>
> <<What do you mean by parser tokens? Do you mean imaginary tokens
> inserted by the parser for which no actual input lexeme exists?>>
> Well,
> why doesn't the docs explain more of this stuff? I still do not
I was actually just wondering what a "parser token" was as *I* hadn't
heard the term and didn't know what he was asking. ;)
> Any programming tool has its own set of programming Idioms, this is
> also
> the case with ANTLR: When I first started off with ANTLR, there where
> two problems that kept me occupied:
>
> 1) How to parse my language
> 2) How to execute it
>
> So what docs where available to help with this: All over the net there
> are examples on how to execute Expressins ( 2+ 3 etc) using Tree
> walking. But how to execute an IF statement, or a LOOP ?
True, the examples are more complicated. The problem you have though
is a general "how do I build an interpreter" problem, which has
nothing to do with ANTLR really. ANTLR is just the structure
recognizer. You need to learn about symbol tables, exec
environments, and a bunch of other stuff. Perhaps my grad course
notes can help you out there.
> The more advanced examples like TINY Basic is for a beginner mind
> boggling. Why not make a minimalistic example that shows BOTH how to
> parse and execute:
>
> a) Expressions
> b) statements
> c) loops
> d) conditionals
> e) execute simple function calls
Sounds great! When you finish with yours, can you post it or some of
it?
> One of the "Patterns" I have found is to create what I call "Holders":
> They are ANTLR Tree Walker subtrees created duing Parsing that do not
> immediately execute when the tree is walked, but can be called to
> do so
> conditionally by calling another Tree Walker Rule that executes the
> Holders. This is at least how I managed conditionals and loops.
>
> Then the docs need explain how to make the language case sensitive or
> not (I still haven't got ANTLR to accept my camel-case grammar with
> keywords like "myKeyword"). Also the concept of
>
> For the Lexer we need to understand how to parse more complex numbers
> like 1e10 etc. How to define a charVocabulary is also important and
> not
> trivial to the beginner.
Yes, this I hope is much easier in 3.0...
> Then it is the issue of reporting Errors and thus to get hold of
> Source
> Code positions. I may have forgotten some details here, but I think I
> had to implement my own ASTFactory to get hold of the source position
> (getColumn() getLine()) for my Tokens (here we go again: What
> Tokens?).
Yes, this was a big mistake in 2.x and the tree parsers essentially
suck at error reporting.
> I also discovered that the concept of Line and Column is NOT very
> usefull. I will try to explain why: At least Java Editors want the
> OFFSET in the text to highlight a position (like we want to do to
> highlight a parsing error). Now, in my case I wanted to hide parts of
> the source to the user: Call it Headers and Footers of my language,
> so I
> parsed out the part I wanted to show to the user, and wanted to report
> error positions releative to this: It then turns out that the
> concept of
> Line and Column breaks down, and that indeed OFFSET is the only
> sensible
> positioning system. So I changed my lexer to only uppdate column and
> never increment line: Thus getColumn() is now the offset!
In 3.0, the column is now char position in line and the token doesn't
have a string--it has indexes into the text buffer start/stop for
that token :)
> There are more problems I have been struggling with, but this is
> what I
> remember right now.
>
> Finally, there is the issue of licensing: Can you PLEASE put up a link
> to the licensing policies that exists for version 3.0?
I give my students a demerit when they ask a quesiton that is on the
first page of google. In this case, "antlr license" reveals the
first entry as your answer:
http://www.antlr.org/license.html
> I know it will be under the BSD license: Where do I find it?
Try google again and the first entry is what you want.
> And, as reading foreign language license polices is not the simplest
> thing in the world, can someone in plain english explain whether
> the BSD
> license still gives me the right to use ANTLR 3.x and onwards for
> commercial usel, or am I know stuck with 2.x
BSD says you can do what you want, but don't sue me. Also, i make no
guarantee that the code won't destroy all life on this planet. ;) It
means you can do anything you want commercially with it. :)
Ter
--
CS Professor & Grad Director, University of San Francisco Creator, ANTLR
Parser Generator, http://www.antlr.org Cofounder, http://www.jguru.com
More information about the antlr-interest
mailing list