Linking code coverage to version control

Question

Before I commit a change, I'd like to be sure that all of it has been tested, either automatically or manually with a code coverage report, but there is lots of legacy code that doesn't have automated tests, and won't be affected by my changes.

Is there a tool that can cross reference a diff from a version control tool with a code coverage report and make sure everything that has been changed has been run?

I realise that with code coverage, this may give a false sense of security, and with something like this, even more so, but I think it would be worth trying. I use git and PHP - I've used XCache's code coverager interface to browse what I have run, and it's useful, but it would be great if something could run automatically at git commit or push time.

Answer 1

For git diffs, there is a tool named diff-cover, which can check coverage. It takes Cobertura XML coverage reports and compares with the output of git diff. It then reports coverage information for lines in the diff.

Given the proper coverage xml file, you can use this command to check the coverage of your changes compared to the master branch:

$ diff-cover coverage.xml

It is also pretty simple to integrate with a CI server, as long as it can provide you with the commit you need to compare with, like $GIT_PREVIOUS_COMMIT in Jenkins.

Answer 2

You could set up a continuous integration build server (there are lots of excellent, free build servers). One of the build steps would be running code coverage. You could set it up to ignore legacy code and calculate coverage only for non-legacy code. Then you set it up to fail the build if coverage < xx%. Or even fail if coverage % decreases from the previous build.