What is the rationale behind the choice of specific kinds of coverage criterias when looking at an algorithm?

Question

So, I am unsure of what are the pros and cons of du-path coverage (or for instance, any data flow coverage criteria) versus predicate criterias or branch/node criterias.

I can see that in certain cases there are clear advantages of one kind of coverage to the other. For instance, if a lot of my algorithm consists in something alike the next example

void m();
void n();

void method(boolean b) {
    if (b) {
        m();
    } else {
        n();
    }
}

it is clear that using any kind of data flow coverage criteria will let a lot of logic untested (which is something we'd like to avoid). Using for the given case a predicate/clause criteria would be way better.

Now, what I'd like to know is for the general case, what are the things you look for in an algorithm when deciding which kind of coverage criteria you'll follow, between

• Graph
• Data Flow
• Logic
• Input
• Syntax

kinds of coverage criteria (basically, the ones found in Introduction to Software Testing). That is, I am basically looking for general heuristics to follow, for the general case.

Thanks

Answer 1

Admittedly, I've not done much research in this area and am not necessarily following you completely (mostly due to my lack of experience in this subject). But hopefully this is not too far off base.

My understanding of Code Coverage has always been that you want to cover every possible execution path. Now I know some paths are much more important than others (for example: the "happy path" is much more important than some obscure path to set some property), but regardless of weather or not you "cover" every path, you at least want to be aware of their existence and make somewhat of a conscious choice on what and what not to cover (via unit tests or manually).

You can start by eyeballing methods and writing testcases, but this quickly becomes unreliable since you're guaranteed NOT to see every possibly execution path no matter the algorithm type. Even very small programs will produce bugs that were unaccounted for since the tester failed to think of "trying it" that way.

What you really need is a Code Coverage tool to tell you which execution paths were covered by your unit-tests and which were not. At that point, you can make logical decisions on weather or not to cover those missing cases (since not all cases may not be worth your time). Without such a tool, I'd think you'd spend countless man hours going line by line and keeping track of what testcases covered what... Even if the tool costs several hundred dollars (or even several thousand dollars), I think you'd quickly recoup those funds in time saved.

So my general heuristic for you is to use such a tool to track your testing coverage and make decisions to cover or not to cover based on its results.

Some code coverage tools (not comprehensive):

• .NET - NCover
• PHP - PHPCoverage
• JAVA - EMMA
• Ruby - CoverMe
• JavaScript - JSCoverage