[antlr-interest] Why no links to ANTLR 3.0 on www.antlr.org??

Terence Parr parrt at cs.usfca.edu
Thu Jun 9 10:35:59 PDT 2005

On Jun 9, 2005, at 5:12 AM, Geir Ove Skjaervik wrote:

> Hello Terrnce,

Hi !

> The way you respond to mr. Blomberg tells me that at your level of
> knowlegde of Parsing and Parser technology you do not any longer
> comprehend the problems beginners are struggling with: You ask  
> questions
> the novice / beginner does NOT know how to answer because (he / she /
> we) are trying to grasp the concept.

Well, I do have to teach students this stuff on a regular  
basis...seems I succeed sometimes...

> Please do take this as constructive feedback as better  
> documentation is
> indeed needed to get the beginners going.

I appreciate all feedback!  I want my doc to be as helpful as  
possible.  I plan on an O'Reilly style ANTLR book first then a  
"language translation" book next.

> I have completed my first small Programming Language in ANTLR that
> exeuctes by Tree Walking,

Good for you!

> and I still do not fully understand what you
> ask Mr. Blomberg... Maybe again it's me...
> E.g. You asked:
> <<What do you mean by parser  tokens?  Do you mean imaginary tokens
> inserted by the parser for  which no actual input lexeme exists?>>  
> Well,
> why doesn't the docs explain more of this stuff? I still do not

I was actually just wondering what a "parser token" was as *I* hadn't  
heard the term and didn't know what he was asking. ;)

> Any programming tool has its own set of programming Idioms, this is  
> also
> the case with ANTLR: When I first started off with ANTLR, there where
> two problems that kept me occupied:
> 1) How to parse my language
> 2) How to execute it
> So what docs where available to help with this: All over the net there
> are examples on how to execute Expressins ( 2+ 3 etc) using Tree
> walking. But how to execute an IF statement, or a LOOP ?

True, the examples are more complicated.  The problem you have though  
is a general "how do I build an interpreter" problem, which has  
nothing to do with ANTLR really.  ANTLR is just the structure  
recognizer.  You need to learn about symbol tables, exec  
environments, and a bunch of other stuff.  Perhaps my grad course  
notes can help you out there.

> The more advanced examples like TINY Basic is for a beginner mind
> boggling. Why not make a minimalistic example that shows BOTH how to
> parse and execute:
> a) Expressions
> b) statements
> c) loops
> d) conditionals
> e) execute simple function calls

Sounds great!  When you finish with yours, can you post it or some of  

> One of the "Patterns" I have found is to create what I call "Holders":
> They are ANTLR Tree Walker subtrees created duing Parsing that do not
> immediately execute when the tree is walked, but can be called to  
> do so
> conditionally by calling another Tree Walker Rule that executes the
> Holders. This is at least how I managed conditionals and loops.
> Then the docs need explain how to make the language case sensitive or
> not (I still haven't got ANTLR to accept my camel-case grammar with
> keywords like "myKeyword").  Also the concept of
> For the Lexer we need to understand how to parse more complex numbers
> like 1e10 etc. How to define a charVocabulary is also important and  
> not
> trivial to the beginner.

Yes, this I hope is much easier in 3.0...

> Then it is the issue of reporting Errors and thus to get hold of  
> Source
> Code positions. I may have forgotten some details here, but I think I
> had to implement my own ASTFactory to get hold of the source position
> (getColumn() getLine()) for my Tokens (here we go again: What  
> Tokens?).

Yes, this was a big mistake in 2.x and the tree parsers essentially  
suck at error reporting.

> I also discovered that the concept of Line and Column is NOT very
> usefull. I will try to explain why: At least Java Editors want the
> OFFSET in the text to highlight a position (like we want to do to
> highlight a parsing error).  Now, in my case I wanted to hide parts of
> the source to the user: Call it Headers and Footers of my language,  
> so I
> parsed out the part I wanted to show to the user, and wanted to report
> error positions releative to this: It then turns out that the  
> concept of
> Line and Column breaks down, and that indeed OFFSET is the only  
> sensible
> positioning system. So I changed my lexer to only uppdate column and
> never increment line: Thus getColumn() is now the offset!

In 3.0, the column is now char position in line and the token doesn't  
have a string--it has indexes into the text buffer start/stop for  
that token :)

> There are more problems I have been struggling with, but this is  
> what I
> remember right now.
> Finally, there is the issue of licensing: Can you PLEASE put up a link
> to the licensing policies that exists for version 3.0?

I give my students a demerit when they ask a quesiton that is on the  
first page of google.  In this case, "antlr  license" reveals the  
first entry as your answer:


> I know it will be under the BSD license:  Where do I find it?

Try google again and the first entry is what you want.

> And, as reading foreign language license polices is not the simplest
> thing in the world, can someone in plain english explain whether  
> the BSD
> license still gives me the right to use ANTLR 3.x and onwards for
> commercial usel, or am I know stuck with 2.x

BSD says you can do what you want, but don't sue me.  Also, i make no  
guarantee that the code won't destroy all life on this planet. ;)  It  
means you can do anything you want commercially with it. :)

CS Professor & Grad Director, University of San Francisco
Creator, ANTLR Parser Generator, http://www.antlr.org
Cofounder, http://www.jguru.com

More information about the antlr-interest mailing list