[antlr-interest] Rule coverage testing

Micke Hovmöller micke.hovmoller at gmail.com
Mon Sep 21 08:21:50 PDT 2009


There is a code coverade plugin in NetBeans, so I tried with that.

It sort of works. My grammar is 656 lines. The resulting parser is 7074
lines, both including whitespace.

The coverage report says that my tests cover 2338 of 2966 lines.
Unfortunately, I can't find out how to break this down by function. The
parser file is color coded per line, but I can't see the difference between
lines counted as whitespace and lines that are not covered but really should
be.

Also, a large amount of the uncovered lines are exception handling, most of
which is automatically generated, and I'm not interested in trying to come
up with test cases for that, if that is even possible in every case.

Finally, some of the stuff doesn't make any sense to me. For instance,
towards the end is this code:
---
    static final String DFA56_eotS =
        "\33\uffff";
    static final String DFA56_eofS =
        "\33\uffff";
    static final String DFA56_minS =
        "\3\4\1\uffff\1\4\23\uffff\1\4\2\uffff";
    static final String DFA56_maxS =
        "\3\u008d\1\uffff\1\u008d\23\uffff\1\u008d\2\uffff";
    static final String DFA56_acceptS =
        "\3\uffff\1\26\1\uffff\1\1\1\2\1\3\1\4\1\5\1\7\1\10\1\11\1\12\1"+
        "\13\1\14\1\15\1\16\1\21\1\22\1\23\1\24\1\25\1\20\1\uffff\1\17\1"+
        "\6";
    static final String DFA56_specialS =
        "\33\uffff}>";
    static final String[] DFA56_transitionS = {
            "\1\3\1\uffff\1\2\2\uffff\1\1\1\15\1\22\2\uffff\1\10\1\4\1\16"+
( This last string goes on for 40 lines)
---

The last static final String line is marked as covered, but not the others.
And it's quite unclear to me if the 40 line string is counted as 1 line, 40
lines (all covered) or 1 covered and 39 uncovered lines. I suspect the
first, but have found no good documentation and no proper reports.

It's possible (no, make that "likely") that I am doing something wrong, and
I would very much appreciate pointers in the right direction.

/Micke

On Fri, Sep 18, 2009 at 7:27 PM, Joshua Ganderson
<joshganderson at gmail.com>wrote:

> Why not just use unit tests that leverage your test files and use a
> coverage plugin to test coverage by class/method when running your tests? I
> don't know about netbeans but eclipse has some nice/free plugins for this.
>
>
> On Fri, Sep 18, 2009 at 6:24 AM, Jim Idle <jimi at temporal-wave.com> wrote:
>
>> Switch to maven, write coverage tests (there are food choices for java
>> ones ) to run against the binary parser and see which methods are
>> never called . Methods are named after rules.
>>
>> Jim
>>
>> On Sep 18, 2009, at 3:58 AM, Micke Hovmöller
>> <micke.hovmoller at gmail.com> wrote:
>>
>> > I've written a reasonably large grammar for handling poker hand
>> > histories. I have worked on and off for about a year or so on this.
>> > During this time, the file format has changed slightly. I suspect
>> > that I now have a bunch of rules in my grammar that are no longer
>> > used.
>> >
>> > What are best practices for establishing rule coverage test cases?
>> > Ideally, I want to have a bunch of infiles, run them through the
>> > grammar and have output displaying which rules were never used.
>> > Showing frequency of usage would be nice, but it is definitely not
>> > needed.
>> >
>> > This is developed in NetBeans 6.7.1 with the ANTLR plugin, so
>> > anything that runs in that environment is much preferred to any
>> > other solution, but I am prepared to switch IDE if that is required.
>> >
>> > /Micke
>> >
>> >
>> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
>> > Unsubscribe:
>> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>>
>> List: http://www.antlr.org/mailman/listinfo/antlr-interest
>> Unsubscribe:
>> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20090921/9fb07e7d/attachment.html 


More information about the antlr-interest mailing list