[antlr-interest] gUnit for v4?

[Kieran Simpson]

On 20/04/12 3:11 AM, Terence Parr wrote:
> Hi Kieran,
>
> thanks very much for the feedback. What exactly were you testing when you say parts of your grammar and in different combinations? Were you testing the ASTs you created? It was very useful for that. Since v4 automatically builds trees, those trees will be correct unless there's a bug so there's nothing really to test as far as I can tell.
> Ter

Ter and list,
   I must admit I haven't been watching the development of v4 that 
closely so perhaps there's something I'm not understanding about AST 
construction/rewrite rules.

An example from my current project is that you can define an entity 
inside a namespace.  However there is the ability to define the 
namespace as part of the entity def (it can make the input more readable).

eg: namespace(namespaceId)  { entity(entityId) ... }

eg: entity(namespaceId:entityId) ....

Conceptually the result is the same, a namespace tree with the entity as 
a subtree.  Using gUnit I've been able to write tests quickly! that test 
whether input combinations not only match, but build the correct AST 
structure.

Coupled with Token testing, and more complicated parser rules I've found 
gUnit valuable to my TDD process as I can come up with a bunch of input 
and refactor my grammar as I go.  Given my target AST walker language is 
C, in my C++ tests (the rest of my app is in C++) I can focus on 
semantic issues.  The separation of concerns testing wise is also nicer 
as I can isolate syntactic issues from semantic problems.

Cheers,

> On Apr 18, 2012, at 9:56 PM, Kieran Simpson wrote:
>
>> On 23/07/64 5:59 AM, Terence Parr wrote:
>>> Hi,
>>>
>>> I started down the path of building the new version of gUnit for v4, but I'm beginning to question its value for the new version. Because I am pooh-poohing putting actions directly within a grammar, grammar can only yield parse trees. and looks nice to be able to say:
>>>
>>> expr:
>>> 	"1"		->   (expr 1)
>>> 	"1+2"	->   (expr (expr 1) + (expr 2))
>>>
>>> but so what? all that did was confirm that the parser generated by ANTLR works correctly. it's not telling you anything you didn't know by looking at the grammar.
>>>
>>> In the old days we used to build ASTs and so we needed to check the structure. That need is no longer here. Also, these rules don't generate output and so we can't check input to output translation.
>>>
>>> What it comes down to is this: we really need functional testing not unit testing for language applications, unless of course each rule returns a string that the translation of the input some phrase or something. But, it's just as easy to write calls to assertEquals() saying that input x should yield input y. If we are doing something other than generating output, such as building up a data structure, then we really do need to manually check that the data structure is solid given some input.
>>>
>>> I'm trying to find a flaw in my logic. Is there something I'm missing about testing grammars? In v4, they will give you exactly what you want, unless there is a bug in ANTLR. But that's my problem not yours.
>>>
>>> If we can't find a major benefit, then we should avoid introducing another moving part to the ANTLR ecosystem.
>>
>> I find gUnit very helpful in testing parts of my grammar both in
>> isolation and different combinations.  I like being able to test token
>> definitions against different input.
>>
>> Where I find functional testing more beneficial is for tree walkers.
>> For my grammars that produce ASTs I've found the grammar tests useful in
>> preventing regression bugs.
>>
>> There's also testing in non Java languages to consider.  Having the
>> ability to have tests generated/run via JUnit may be easier than having
>> to piece together your tests in another language (eg: C) which I
>> personally find is more time consuming.
>>
>> Just a few thoughts.
>>
>> Cheers,