What type of errors could my code still contain even if I have 100% code coverage?

Question

What type of errors could my code still contain even if I have 100% code coverage? I'm looking for concrete examples or links to concrete examples of such errors.

Answer 1

Having 100% code coverage is not that great as one may think of it. Consider a trival example:

double Foo(double a, double b)
{
    return a / b;
}

Even a single unit test will raise code coverage of this method to 100%, but the said unit test will not tell us what code is working and what code is not. This might be a perfectly valid code, but without testing edge conditions (such as when b is 0.0) unit test is inconclusive at best.

Code coverage only tells us what was executed by our unit tests, not whether it was executed correctly. This is an important distinction to make. Just because a line of code is executed by a unit test, does not necessarily mean that that line of code is working as intended.

Listen to this for an interesting discussion.

Answer 2

Code coverage doesn't mean that your code is bug free in any way. It's an estimate on how well you're test cases cover your source code base. 100% code coverage would imply that every line of code is tested but every state of your program certainly is not. There's research being done in this area, I think it's referred to as finite state modeling but it's really a brute force way of trying to explore every state of a program.

A more elegant way of doing the same thing is something referred to as abstract interpretation. MSR (Microsoft Research) have released something called CodeContracts based on abstract interpretation. Check out Pex as well, they really emphasis cleaver methods of testing application run-time behavior.

I could write a really good test which would give me good coverage, but there's no guarantees that that test will explore all the states that my program might have. This is the problem of writing really good tests, which is hard.

Code coverage does not imply good tests