[antlr-interest] Why no links to ANTLR 3.0 on www.antlr.org??
parrt at cs.usfca.edu
Thu Jun 9 10:35:59 PDT 2005
On Jun 9, 2005, at 5:12 AM, Geir Ove Skjaervik wrote:
> Hello Terrnce,
> The way you respond to mr. Blomberg tells me that at your level of
> knowlegde of Parsing and Parser technology you do not any longer
> comprehend the problems beginners are struggling with: You ask
> the novice / beginner does NOT know how to answer because (he / she /
> we) are trying to grasp the concept.
Well, I do have to teach students this stuff on a regular
basis...seems I succeed sometimes...
> Please do take this as constructive feedback as better
> documentation is
> indeed needed to get the beginners going.
I appreciate all feedback! I want my doc to be as helpful as
possible. I plan on an O'Reilly style ANTLR book first then a
"language translation" book next.
> I have completed my first small Programming Language in ANTLR that
> exeuctes by Tree Walking,
Good for you!
> and I still do not fully understand what you
> ask Mr. Blomberg... Maybe again it's me...
> E.g. You asked:
> <<What do you mean by parser tokens? Do you mean imaginary tokens
> inserted by the parser for which no actual input lexeme exists?>>
> why doesn't the docs explain more of this stuff? I still do not
I was actually just wondering what a "parser token" was as *I* hadn't
heard the term and didn't know what he was asking. ;)
> Any programming tool has its own set of programming Idioms, this is
> the case with ANTLR: When I first started off with ANTLR, there where
> two problems that kept me occupied:
> 1) How to parse my language
> 2) How to execute it
> So what docs where available to help with this: All over the net there
> are examples on how to execute Expressins ( 2+ 3 etc) using Tree
> walking. But how to execute an IF statement, or a LOOP ?
True, the examples are more complicated. The problem you have though
is a general "how do I build an interpreter" problem, which has
nothing to do with ANTLR really. ANTLR is just the structure
recognizer. You need to learn about symbol tables, exec
environments, and a bunch of other stuff. Perhaps my grad course
notes can help you out there.
> The more advanced examples like TINY Basic is for a beginner mind
> boggling. Why not make a minimalistic example that shows BOTH how to
> parse and execute:
> a) Expressions
> b) statements
> c) loops
> d) conditionals
> e) execute simple function calls
Sounds great! When you finish with yours, can you post it or some of
> One of the "Patterns" I have found is to create what I call "Holders":
> They are ANTLR Tree Walker subtrees created duing Parsing that do not
> immediately execute when the tree is walked, but can be called to
> do so
> conditionally by calling another Tree Walker Rule that executes the
> Holders. This is at least how I managed conditionals and loops.
> Then the docs need explain how to make the language case sensitive or
> not (I still haven't got ANTLR to accept my camel-case grammar with
> keywords like "myKeyword"). Also the concept of
> For the Lexer we need to understand how to parse more complex numbers
> like 1e10 etc. How to define a charVocabulary is also important and
> trivial to the beginner.
Yes, this I hope is much easier in 3.0...
> Then it is the issue of reporting Errors and thus to get hold of
> Code positions. I may have forgotten some details here, but I think I
> had to implement my own ASTFactory to get hold of the source position
> (getColumn() getLine()) for my Tokens (here we go again: What
Yes, this was a big mistake in 2.x and the tree parsers essentially
suck at error reporting.
> I also discovered that the concept of Line and Column is NOT very
> usefull. I will try to explain why: At least Java Editors want the
> OFFSET in the text to highlight a position (like we want to do to
> highlight a parsing error). Now, in my case I wanted to hide parts of
> the source to the user: Call it Headers and Footers of my language,
> so I
> parsed out the part I wanted to show to the user, and wanted to report
> error positions releative to this: It then turns out that the
> concept of
> Line and Column breaks down, and that indeed OFFSET is the only
> positioning system. So I changed my lexer to only uppdate column and
> never increment line: Thus getColumn() is now the offset!
In 3.0, the column is now char position in line and the token doesn't
have a string--it has indexes into the text buffer start/stop for
that token :)
> There are more problems I have been struggling with, but this is
> what I
> remember right now.
> Finally, there is the issue of licensing: Can you PLEASE put up a link
> to the licensing policies that exists for version 3.0?
I give my students a demerit when they ask a quesiton that is on the
first page of google. In this case, "antlr license" reveals the
first entry as your answer:
> I know it will be under the BSD license: Where do I find it?
Try google again and the first entry is what you want.
> And, as reading foreign language license polices is not the simplest
> thing in the world, can someone in plain english explain whether
> the BSD
> license still gives me the right to use ANTLR 3.x and onwards for
> commercial usel, or am I know stuck with 2.x
BSD says you can do what you want, but don't sue me. Also, i make no
guarantee that the code won't destroy all life on this planet. ;) It
means you can do anything you want commercially with it. :)
CS Professor & Grad Director, University of San Francisco
Creator, ANTLR Parser Generator, http://www.antlr.org
More information about the antlr-interest