[antlr-interest] announcing gUnit: ANTLR grammar unit testing

[Johannes Luber]

Terence Parr wrote:
> Howdy.  Martin Bravenboer gave me a look at his nice grammar testing
> facilities for a GLR-based parser generator (ASF+SDF) at OOPSLA last
> year.  He inspired me to build one for ANTLR.  Fortunately, I found a
> very capable graduate student, Leon Su, that was able to implement it
> this summer.  Here is the wiki with code attached. :)
> 
> http://www.antlr.org/wiki/display/ANTLR3/gUnit+-+Grammar+Unit+Testing
> 
> Please direct feedback to the group and/or Leon so we can improve it.  I
> think you'll find it is a very nice tool.  Has both interpreted and
> gUnit->jUnit code gen modes. :)
> 
> Terence

I haven't tested the the program myself, so I comment on what I've read
on the wiki.

The first thing I noticed is that you don't explain the test suite
example. Someone with no knowledge of gUnit won't be able to decipher
your example. Look at those lines:

variable: // test rule:variable with 2 tests
"int x" FAIL

"int x" -> ""

What is meant with both results? How can they differ for the same input?
Or do you want to demonstrate two abilities of gUnit? And what is the
meaning of the '->'? (The mention in the "gUnit grammar" section is too
far away. And what kind of output do you mean? System.out.println()
calls? Or the result of a StringTemplate?)

Then you show us the result of the test run. I noticed that you number
the tests internally, but there is no visible mapping for the human eye.
Suppose a file with several hundreds of tests. How do you know which one
is the 186th one? You show only the grammar rule but the actual culprit
has still to be inferred.

Another problem is that the test results aren't that visibly distinctive
from each other. There aren't signature characters to scan for. I'd like
to see even such characters for the expected input and output lines so
it isn't to confusing if the output goes over several lines. It makes
also easier to discriminate between the gUnit program output and the
test suite outputs. NUnit has a nice syntax for it. Maybe you can use
theirs?

The last point: Are other target languages also supported? Or do only
Java grammars work for now?

Best regards,
Johannes Luber