difference between code coverage and profiling

Question

What is difference between code code coverage and profiling.

Which is the best open source tool for code coverage.

Answer 1

Code coverage is an assessment of how much of your code has been run. This is used to see how well your tests have exercised your code.

Profiling is used to see how various parts of your code perform.

The tools depend on the language and platform you are using. I'm guessing that you are using Java, so recommend CodeCover. Though you may find NoUnit easier to use.

Answer 2

Coverage is important to see which parts of the code have not been run. In my experience it has to be accumulated over multiple use cases, because any single run of the software will only use some of the code.

Profiling means different things at different times. Sometimes it means measuring performance. Sometimes it means diagnosing memory leaks. Sometimes it means getting visibility into multi-threading or other low-level activities.

When the goal is to improve software performance, by finding so-called "bottlenecks" and fixing them, don't just settle for any profiler, not even necessarily a highly-recommended or venerable one. It is essential to use the kind that gets the right kind of information and presents it to you the right way, because there is a lot of confusion about this. More on that subject.

Added: For a coverage tool, I've always done it myself. In nearly every routine and basic block, I insert a call like this: Utils.CovTest("file name, routine name, comment that tells what's being done here"). The routine records the fact that it was called, and when the program finishes, all those comments are appended to a text file. Then there's a post-processing step where that file is "subtracted" from a complete list (gotten by a grep-like program). The result is a list of what hasn't been tested, requiring additional test cases.

When not doing coverage testing, Utils.CovTest does nothing. I leave it out of the innermost loops anyway, so it doesn't affect performance much. In C and C++, I do it with a macro that, during normal use, expands to nothing.