[antlr-interest] unit tests

Emond Papegaaij e.papegaaij at student.utwente.nl
Fri Nov 24 00:05:31 PST 2006


On Thursday 23 November 2006 23:33, Robert Hill wrote:
<cut HOWTO test parser generator output>
> I was scratching my head a little, wondering how you'd actually test your
> output. I ended up with the output file method purely because I can't
> really see any other non-intrusive way to test the output. If the tests
> were intrusive then I guess removing the tests would by definition require
> retesting..lol! My only worry is minor formatting changes would make tests
> fail when it doesn't really affect the files correctness. I suppose I could
> strip out all the whitespace, hmm I'll have another think about this!
>
> Thanks for the pointers that I'm on the right track :)
>
> I'll check out the antlr tests, and the sun methods too.

Do you really want to test the output of ANTLR? What exactly do you want to 
test? In my opinion, you should specify your grammar in EBNF and create your 
parser directly from this EBNF specification. The only thing you have to 
test, is whether your parser specification matches your grammar 
specification. But as ANTLR supports EBNF grammars, this proof is almost 
trivial. Whatever you do, try not to test ANTLR itself. There is no point in 
testing whether ANTLR generates the correct code for a parser specification. 
This is the job of the ANTLR test cases. If you go down this road, where does 
it end? Do you need to test whether the JVM really gives 2 for 1+1?

Ofcourse, it can help to have several correct inputs, to test if your grammar 
really accepts the language you intended. I've never found incorrect inputs 
to be very useful. Normally you'd want to use a cornercase as incorrect 
input. But what is a cornercase for a grammar? Failing tests usually end up 
testing the parser generator, not your grammar.

There is an exception to this: non-trivial semantic predicates. In my opinion 
these need both passing and failing tests.

Comparing the output of ANTLR to a copy of that file is not a very good test. 
How do you know that the output, you compare against, is correct? Does a 
change in the file really indicate a problem, or just another way to solve 
the same problem? To test a generated grammar, simply run it with a sample 
input and check the output. The output of a parser depends on how you use the 
parser. It can be an AST (you can check those by using toStringTree()), a 
simple pass or fail or something else.

Best regards,
Emond



More information about the antlr-interest mailing list