[antlr-interest] soliciting language recipes book outline feedback

Harald M. Müller harald_m_mueller at gmx.de
Tue Jan 1 02:39:32 PST 2008


Hi -

At minimum, I would require that there are two solutions for many (? - a few
important) of the examples. In the real world, there is not "the solution."
(and of course, not both/more solutions need to be done down to code ... but
there should - like in any good patterns book - be enough meat so that the
problems are made clear when solving one over the other solution; I'm very
convinced that there is never ever a "perfect solution" in your business).

Just to lay out that "design space" a little: Whether you 

* do your work in the parser;
* create an AST and do your work there;
* create an AST, rewrite in to another AT, do your work there;
* create a complete different structure (e.g. a flow graph [think "byte
code"]) to do your work; 
* create an AST, the create the different structure and do your work there;

depends very much 

* on your current needs ("semantics") (are there "tree jumping semantics" =
work on some distant, even upcoming subtree; "flow jumping semantics" = do
something in the middle of some distant, even umpcoming tree; "simple
semantics" = work recursively down/up the AST);
* on future needs (are you going to add additional semantics to the tree -
optimizations [think constant folding], documentation [think Javadoc], ...)

Before I ever start laying out a grammar, I write down this design space for
a tool - which sometimes results in creating an AST at a time where this
seems "way to complicated" - but I know it will help with that "tiny
language extension" they are already talking about ... or the opposite:
Seeing that a generalization (even a small one) requires much work, I reduce
the machine to a "small one" (and will bill the customer for the large one
when he really claims to need it ...).

Second, there should probably be advice on when/where to stop using ANTLR.
The two "boundaries" that right now come to mind are
* XML - you can do something with ANTLR here (see a few great Wiki entries),
but of course there are lots of XML-specific technologies;
* rewriting - when a rewriting solution is going to become a "calculus,"
other tools (one was mentioned here a few days ago ...) might be a better
choice - but why?

.... just my two cents (also why I find today's wiki entries somewhat ad-hoc
...).

Regards
Harald

> -----Original Message-----
> From: antlr-interest-bounces at antlr.org 
> [mailto:antlr-interest-bounces at antlr.org] On Behalf Of Terence Parr
> Sent: Monday, December 31, 2007 10:52 PM
> To: antlr-interest at antlr.org List
> Cc: Susannah Davidson Pfalzer
> Subject: [antlr-interest] soliciting language recipes book 
> outline feedback
> 
> Howdy folks,
> 
> I've signed up to do another book for pragmatic programmers.  
> This time I'm going to do a collection of "how to" 
> examples/recipes.  The book is meant to teach you how to fish 
> not give you fish--the examples will be necessarily smallish. 
> I anticipate the level being introductory through 
> intermediate as real, big examples will simply not fit. I'd 
> like to start out with really brain-dead examples such as  
> comma separated value parsing so that I can explain the 
> fundamentals.   
> I will always remember the first technical book I read (on 
> 6502 assembly code code by Lance Levanthal).  This awesome 
> book started out showing how to clear a memory location. This 
> gave me the confidence that I might actually understand the 
> rest of the book :)
> 
> I propose breaking the book up into three main sections: 
> parsing, interpretation, translation. I would love to hear 
> peoples feedback about what examples would be useful and 
> exciting.  Keep in mind that the average example will need to 
> fit in about 5 to 10 pages. Code samples will be made 
> available online and not necessarily completely printed out 
> in the book to save space. Also I would love to hear your 
> feedback about how the recipes should be laid out or described.
> 
> Please CC Susannah Pfalzer, with whom I'm fortunate to work 
> with again, on your replies as she is not a member of this list. :)
> 
> Here is my rough draft list of examples, which is a bit too few.   
> Thanks and happy new year!
> 
> Ter
> ------------
> I. Parsing recipes
> 	comma-separated values (show in java then python or 
> ruby then ANTLR)
> 	parsing binary data (fixed and non-fixed size records)
> 	parsing log files (in awk/sed/grep then ANTLR)
> 	configuration file or JSON or YAML
> 	text-based network protocol (e.g., POP, nntp, http)
> 	graphs and networks via edge list format
> 	read fixed a XML format like:
> 		SVG (vector graphics)
> 		or, RDF ontology graph
> 	handling include files
> 	parsing island grammars like JSP
> 	simplified English imperative statement recognizer
> 		(E.g., games: "pick up gun and shoot spy")
> 
> II. Interpretation recipes
> 	arithmetic expression trees
> 	polynomial differentiation
> 	query language (parse and pull data from DB)
> 	serial/deserialize objects
> 	partial PS interpreter
> 	simple programming language interpreter
> 
> III. Translation recipes
> 	wiki to HTML
> 	simplified programming language pretty printer
> 	instrumenting simplified programming language with 
> profiling info
> 	simple Java byte code generator
> 	simple robot control language for lego MindStorm robots
> 	class hierarchy generator (awk/sed/grep then grep+ANTLR)
> 		generates DOT format for graphviz to visualize
> 



More information about the antlr-interest mailing list