[antlr-interest] gUnit for v4?

[Terence Parr]

On Apr 30, 2012, at 7:38 PM, David Meibusch wrote:

> Ter,
> 
> I can layout out our use case, just to see if it confirms your
> assumptions.

Hi David, thanks very much for your response! I believe it's still okay to drop gUnit for v4…

one of your biggest tests is the AST construction, which of course is no longer necessary since ANTLR generates parse trees and automatically.

Your point about worrying about upgrades to the tool causing regressions is well taken, though, the parse tree is a well-defined structure derived from the grammar rules. I suppose in the future the types of the nodes could change, but the "toString" of the payloads would not, unless of course you yourself changed what the payload was. We should be okay here too.

Your grammar uses embedded actions, which we are also strongly discouraging in v4. That means it's also discouraged to return values from rules. So, there's no return values to test from a gUnit tool.

Going through your improvements with respect to v4:

> Expectations have:
> 
> OK/FAIL, same as before

I think this is easy enough to simulate where the sequence of method calls:

String[] inputs = { "input1", "input2", "input3", … };
checkNoErrors(inputs); // checks number of syntax errors from parser

I do this a lot myself in v4. It's very useful when checking that *my* understanding of the grammar and how it assigns precedence to my input streams, but again I don't really need gunit:

		String[] tests = {
			"a",			"(s (e a) <EOF>)",
			"a+b",			"(s (e (e a) + (e b)) <EOF>)",
			"a*b",			"(s (e (e a) * (e b)) <EOF>)",
			"a?b:c",		"(s (e (e a) ? (e b) : (e c)) <EOF>)",
			"a=b=c",		"(s (e (e a) = (e (e b) = (e c))) <EOF>)",
			"a?b+c:d",		"(s (e (e a) ? (e (e b) + (e c)) : (e d)) <EOF>)",
			"a?b=c:d",		"(s (e (e a) ? (e (e b) = (e c)) : (e d)) <EOF>)",
			"a? b?c:d : e",	"(s (e (e a) ? (e (e b) ? (e c) : (e d)) : (e e)) <EOF>)",
			"a?b: c?d:e",	"(s (e (e a) ? (e b) : (e (e c) ? (e d) : (e e))) <EOF>)",
		};
		runTests(grammar, tests, "s");

It's not is pretty, but we get to avoid creating and maintaining a tool. 

> RETURN[<name>] allows you to test a rule return by name

As we are discouraging return values, this has less utility. Of course, if you do in fact have returned items, this would be useful. Whether the return on the investment of creating and maintaining a separate tools worth it, I'm not sure.

> OUTPUT (or ->) allows you to test the parse tree created with no parse
> errors expected

ANTLR should be generating the correct parse tree should be invariant for the same grammar, regardless of upgrades to the tool.

> ERROR allows you to test the parse tree with error nodes expected

I think raw code would also work here. I do something that looks almost like gUnit:

	String[] bad_inlineChecks = {
		"$lab",			"error(67): A.g4:7:4: missing attribute access on rule reference lab in $lab\n",
		"$q",           "error(63): A.g4:7:4: unknown attribute reference q in $q\n",
		"$q.y",         "error(63): A.g4:7:4: unknown attribute reference q in $q.y\n",
		"$q = 3",       "error(63): A.g4:7:4: unknown attribute reference q in $q\n",
		"$q = 3;",      "error(63): A.g4:7:4: unknown attribute reference q in $q = 3;\n",
		"$q.y = 3;",    "error(63): A.g4:7:4: unknown attribute reference q in $q.y = 3;\n",
		"$q = $blort;", "error(63): A.g4:7:4: unknown attribute reference q in $q = $blort;\n" +
						"error(63): A.g4:7:9: unknown attribute reference blort in $blort\n",
		"$a.ick",       "error(65): A.g4:7:6: unknown attribute ick for rule a in $a.ick\n",
		"$a.ick = 3;",  "error(65): A.g4:7:6: unknown attribute ick for rule a in $a.ick = 3;\n",
		"$b.d",         "error(64): A.g4:7:6: cannot access rule d's parameter: $b.d\n",  // can't see rule ref's arg
		"$d.text",      "error(63): A.g4:7:4: unknown attribute reference d in $d.text\n", // valid rule, but no ref
		"$lab.d",		"error(64): A.g4:7:8: cannot access rule d's parameter: $lab.d\n",
	};

> Note that error reporting and recovery is particularly important - in
> fact close to the entire rationale for this parser's existence.

Yep, makes sense. I think we can still go with methods for now. In the end, we might find some good uses for a simple gUnit translator to jUnit cases. But, we can add that later. I'm just trying to figure out what I get into the 1st release.

Ter