[antlr-interest] unit tests

Jeremy D. Frens jdfrens at calvin.edu
Sat Dec 9 10:02:13 PST 2006


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Sorry to jump into the discussion late.  It's too easy to let things
slide while on a sabbatical...

Robert Hill wrote:
> Hi all, just want to open up a bit of a discussion, how does everyone else
> handle unit testing for a compiler/code generator.
> Or if you're using JUnit/NUnit how do test your Antlr output ?
> 
> File compares on the output sources? How do you test individual constructs?
> Hundred of files( or embedded strings) with each one having a single example
> to the construct in question?
> 
> Just looking for some good ideas ( or examples :) ) on implementation (I'll
> be using JUinit btw)

I believe very strongly in unit testing, so while I was writing an
interpreter using ANTLR (http://nolatte.sourceforge.net), I also built a
unit-testing library for ANTLR grammars using JUnit.  My results can be
found on SourceForge: http://antlr-testing.sourceforge.net/ .

Unfortunately, there are a few problems with the ANTLR Testing library:
first, the website sucks.  Second, the library works (as far as I know)
only with ANTLR 2.7.6, and it is unaware of JUnit 4.x.  Third,
ironically, the library is not well tested.  Fourth, I've learned a lot
about unit testing since I wrote the library, and the overall design of
the library could stand some reworking.

As an example of the fourth problem, in order to test a tree parser, you
end up testing the lexer and the parser as well.  I don't know that I'll
necessarily drop this because it *is* good functional testing.  But a
unit test should just test *one* action (e.g., one tree transformation)
in *one* module (e.g., the tree builder).

That's the bad news; here's the good news.

The library is very usable now, even with my warnings.  I've used it for
my own interpreter; I've used it two or three times in a compiler course
(at Calvin College, http://cs.calvin.edu/); I've used it twice in a
Programming Languages course (also at Calvin).

Better yet, the purpose of my sabbatical next semester is to write a lab
manual (maybe textbook) for a Programming Languages course, doing
incremental, test-driven development of interpreters (plural!) and other
language transformations (like a compiler).  It's becoming clear to me
this semester that I need a better ANTLR Testing library to make this
lab manual effective.

Some changes that I've planned (not in any real order): switch to JUnit
4.1 (or whatever's current); rework the existing library for ANTLR
2.7.6; build a new library for ANTLR 3.0; unit test the library; write a
better website.

I'm very interested in what people think about my ANTLR Testing library
and what features you'd like to see in such a library.

jdf

- --
* Jeremy D. Frens * Professor, Computer Science * jdfrens at calvin.edu *
              ``The first step towards wisdom is calling
                things by their right names.''  --Chinese proverb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFevokOcBu2deY79IRAudAAJ9Wkc/dAE0ZOjfQ7f840wmTHCLLaACfd/N4
O3uubpYvv2Ivn6yLrlO7AdE=
=wf3Q
-----END PGP SIGNATURE-----


More information about the antlr-interest mailing list