[antlr-interest] Documenting grammars

[Sam Barnett-Cormack]

Dennis Brothers wrote:
> On Mar 23, 2009, at 2:23 PM, Sam Barnett-Cormack wrote:
> 
>> Sam Harwell wrote:
>>> Why not create our own format that properly describes grammars?
>>>
>>> We could group them by the Tokens file they reference to cover
>>> lexer/parser/tree parser combinations. Documentation could include
>>> formatted/highlighted rule text, comments, DFA statistics, and
>>> thumbnails of the various rule diagrams linked to full-size versions.
>>>
>>> Sam (Harwell)
>> The way I see it, right now, there's two options to get the minimum  
>> I'd
>> like to see:
>>
>> 1) Scratch-written system to document anything, flexibly enough to  
>> allow
>> various different terminologies (appropriate to OOP, grammars,  
>> whatever)
>>
>> 2) Scratch-written grammar documenting system, allowing focus on good
>> documentation of grammars and including the stuff that Sam H talks  
>> about
>> (configurably, of course).
>>
>> Option (2) is less work, while option (1) is more use to the wider
>> community *iff* it's done well. Frankly, I'm leaning towards (2) now
>> (despite some quickly-scratched-out design for (1), which I could  
>> always
>> use to do that option later myself if I really want to). (2) doesn't
>> overlap other existing systems.
>>
>> If there's enthusiasm for this, I'll whip up a quick outline-design,  
>> and
>> anyone who wants to help can help me nail it down to something  
>> specific,
>> and possibly help actually write it ;) areas that'd speed me up most
>> would be writing output engines and bringing extra (more experienced?)
>> minds to the parsing. Design usually produces better results from
>> multiple minds, too.
>>
>> Sam (Barnett-Cormack)
> 
> Since ANTLR uses ANTLR to parse itself, couldn't the ANTLR grammar be  
> modified or extended to recognize and emit doc comments and the  
> elements they refer to?  Seems like this would be quite a leg up on  
> Option 2.

Well, what format would it emit them in? We'd want to have them able to 
be output to html, maybe latex, maybe docbook, maybe $format in future. 
That means some sort of abstract document model. We'd be talking about 
adding on to the ANTLR grammar to emit objects fitting such a model, and 
building an API and some tools to work with this model - and to also 
interface with ANTLR's ability to generate diagrams and suchlike.

As a plus, it's possible that the abstract document model will also be 
appropriate for other parser generators and so on.


-- 
Sam Barnett-Cormack