Code dependency unit testing

Question

In my workplace we have a c# .net solution containing about 50 projects and about 2000 unit tests. After changing code it is required that we run all tests in the solution before pushing our changes to the build server. Running all the tests can take about 10-15 minutes. I was thinking - hey, maybe there could be some process that would analyze all the code changes I made, and then decide to run only the relevant test methods. If such kind of analysis is possible, then instead of running 2000 tests the computer would run only 15 or a 100 tests, then we are talking about a very nice increase in productivity and fewer headaches. Anyone have any ideas on how to implement this, or if such software is available?

Answer 1

It's really difficult to tell what depends on what in a multi-project environment. Ideally:

• Your unit tests would be true unit tests, and 2000 of them should run in maybe 20 seconds.
• Your longer-running tests would be set up on a continuous build on your build server, so you would still get notified pretty fast that you messed things up if you missed anything, but you could personally afford to be just a little more selective in running only the tests that you think you might have affected with your code changes before you commit.

Answer 2

Our C# Test Coverage Tool can do this.

1. It incrementally instruments just the files that changed since the last test coverage run.

2. It can be used to track test coverage for each unit test, and which files are involved. When a file changes, it compares it to the previous version; if a method involved in some unit tests previously changes, it identifies those unit tests. You have to organize that process to associates test runs with coverage data, and re-run the unit tests identified, buts that a small matter of scripting. The reason for this is to enable the test coverage tool to operate with arbitrary testing frameworks.

You can do that on the build server to save time. More importantly, you can provide each developer with this, so that he can run just the tests required to validate the code he changes, before he checks in changes.