Harvey Mudd CS Clinic Project: Differential Code Coverage tool
In 2004, I was given the opportunity to be the customer for a group
of four Harvey Mudd CS students in a yearlong clinic project.
Wine's conformance test suite was only a year or two old, and
I thought it would be useful to create a tool that told us what
areas of Wine were used by real applications but not yet tested by
the conformance suite. So I asked the students to create one.
First they ran the Wine test suite by itself, and wrote one missing
test, to get familiar with the problem. Their patch was accepted, yay!
Then they showed how to run gcov with Wine, and submitted a patch to make it
easier. The first code patch was not accepted, but a documentation patch
and a later code patch were, yay!
They also modified genhtml (part of lcov, a report writer for gcov) to
do nice reports showing which areas of Wine were used by an app
but not the test suite:
In the end, the project had much less impact than I had hoped.
It is easier to run gcov on wine, but lack of coverage just wasn't
a big problem for Wine at the time (though it might be someday).
So, sadly, it was an idea ahead of its time,
and we should simply have focused on writing tests rather than
a magical tool to tell us which tests to work on.
Documents