[antlr-interest] Documenting grammars

[Sam Barnett-Cormack]

Loring Craymer wrote:
> Sam--
> 
> I have to disagree with Andy because I have seen your list of
> requirements on another list.  There was this mad progammer, see, who
> was just starting work on an application he was going to call
> Doxygen.  I wonder if that ever went anywhere? :=)

Doxygen suffering from the same limitation as Javadoc - being solidly 
based on a single paradigm. It also seems poorly documented - I've found 
one fairly huge inconsistency in the docs already, related to adding new 
languages.

> The real question to ask before going off and building another
> doxygen is "How does one document a grammar?"  That leads to "Ok,
> what annotations do I need for comments?" and then to the real
> requirements.  Grammars deal with syntax, syntactic transformation,
> and capturing and expressing semantics; imperative languages deal
> with input/output variables and side effects.  The development
> patterns are different; I have never worried about documenting a
> grammar simply because production names tend to capture input
> semantics, and that usually seems to be enough--grammars tend to be
> compact and reasonably self-documenting.

Well, my combined grammar for ASN.1 has over 1000 lines, isn't properly 
annotated for someone else to understand anyway, and has over 300 
productions - not the most self-documenting thing on Earth, and I don't 
see how it could be.

> Anyway, if you can figure out a set of reasonable documentation
> practices for grammars, please post them to the list for discussion.
> We might learn something.

I'll be doing my initial design work (over and above the vague structure 
in my head) this weekend - I'll certainly post it to the list.

Sam

> ----- Original Message ----
>> From: Sam Barnett-Cormack <s.barnett-cormack at lancaster.ac.uk> To:
>> Andy Tripp <antlr at jazillian.com> Cc: ANTLR Interest Mailing List
>> <antlr-interest at antlr.org> Sent: Thursday, March 26, 2009 8:38:31
>> AM Subject: Re: [antlr-interest] Documenting grammars
>> 
>> Andy Tripp wrote:
>>> I don't think there's any reasonable way to build something that
>>> general. Something that takes any language (e.g. Java, BNF,
>>> assembly, English, etc) and produces any output (HTML, pdf,
>>> postscript, etc) will just be so general as to be useless, IMO.
>>> 
>>> I think you'll end up "just" building a doclet-like API that
>>> handles BNF-like grammars (as opposed to the Java code that
>>> doclets handle). Then create a tool for each type of output,
>>> starting with one that produces HTML (and that's probably the
>>> only one anyone will ever create).
>> Actually, I've already worked out a way of specifying different 
>> hierarchical language-element structure.
>> 
>> For example, Java has packages, which contain interfaces, classes, 
>> enumerations, and annotations. Classes contain fields, methods,
>> classes, interfaces, and enumerations (and possibly annotations).
>> Fields and methods don't contain anything (looking simply), and so
>> on.
>> 
>> ANTLR has grammar "groups" (shared token lists), grammars, rules,
>> etc...
>> 
>> StringTemplate has template groups, templates... maybe add a 
>> higher-level construct purely for documentation purposes.
>> 
>> You build a structure for defining those, and the "middle-man" code
>>  builds a runtime-dependent data structure from the input classes
>> and passes it to the output classes with the relevant element
>> names. The input and output classes *will* have to be aware of
>> possible element types - there's really no way around that. But it
>> means you can easily bolt together new language support just by
>> writing input and output classes, without worrying about the logic
>> that has to between them (which isn't really trivial). And if you
>> add a bit more clever logic, it becomes possible to produce unified
>> documentation that, with one "group" of webpages documenting you
>> grammars, your templates, and your classes. Doesn't sound too
>> general to be useful to me.
>> 
>> Sam
>> 
>>> Sam Barnett-Cormack wrote:
>>>> Okay, this weekend I plan to start work on a suitable 
>>>> language-and-paradigm-independent documentation system, with
>>>> pluggable inputs and outputs. I've not decided what
>>>> technologies to use initially, although you can bet that any
>>>> input and output bits I write will use ANTLR and/or ST,
>>>> appropriately. I've decided to use three languages/paradigms to
>>>> keep in mind to help ensure agnosticism:
>>>> 
>>>> * Grammars (per ANTLR) * Templates (per ST) * Object-Orients
>>>> languages generally (which will probably cover procedural stuff
>>>> as well, really)
>>>> 
>>>> I'll start with the "middle" part - working out an API for
>>>> receiving and outputing language information.
>>>> 
>>>> If there's interest in this, I can either keep the list
>>>> updated, or just those who are interested. If I get it off the
>>>> ground and ANTLR folks don't want it in the ANTLR stable, I'll
>>>> sort out other hosting and mailing list stuff.
>>>> 
>>>> Sam
>> 
>> List: http://www.antlr.org/mailman/listinfo/antlr-interest 
>> Unsubscribe: 
>> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>> 
> 
> 
> 
>