Off By One errors and Mutation Testing

Question

In the process of writing an "Off By One" mutation tester for my favourite mutation testing framework (NinjaTurtles), I wrote the following code to provide an opportunity to check the correctness of my implementation:

public int SumTo(int max)
{
    int sum = 0;
    for (var i = 1; i <= max; i++)
    {
        sum += i;
    }
    return sum;
}

now this seems simple enough, and it didn't strike me that there would be a problem trying to mutate all the literal integer constants in the IL. After all, there are only 3 (the 0, the 1, and the ++).

WRONG!

It became very obvious on the first run that it was never going to work in this particular instance. Why? Because changing the code to

public int SumTo(int max)
{
    int sum = 0;
    for (var i = 0; i <= max; i++)
    {
        sum += i;
    }
    return sum;
}

only adds 0 (zero) to the sum, and this obviously has no effect. Different story if it was the multiple set, but in this instance it was not.

Now there's a fairly easy algorithm for working out the sum of integers

sum = max * (max + 1) / 2;

which I could have fail the mutations easily, since adding or subtracting 1 from either of the constants there will result in an error. (given that max >= 0)

So, problem solved for this particular case. Although it did not do what I wanted for the test of the mutation, which was to check what would happen when I lost the ++ - effectively an infinite loop. But that's another problem.

So - My Question: Are there any trivial or non-trivial cases where a loop starting from 0 or 1 may result in a "mutation off by one" test failure that cannot be refactored (code under test or test) in a similar way? (examples please)

Note: Mutation tests fail when the test suite passes after a mutation has been applied.

Update: an example of something less trivial, but something that could still have the test refactored so that it failed would be the following

public int SumArray(int[] array)
{
    int sum = 0;
    for (var i = 0; i < array.Length; i++)
    {
        sum += array[i];
    }

    return sum;
}

Mutation testing against this code would fail when changing the var i=0 to var i=1 if the test input you gave it was new[] {0,1,2,3,4,5,6,7,8,9}. However change the test input to new[] {9,8,7,6,5,4,3,2,1,0}, and the mutation testing will fail. So a successful refactor proves the testing.

Answer 1

I think with this particular method, there are two choices. You either admit that it's not suitable for mutation testing because of this mathematical anomaly, or you try to write it in a way that makes it safe for mutation testing, either by refactoring to the form you give, or some other way (possibly recursive?).

Your question really boils down to this: is there a real life situation where we care about whether the element 0 is included in or excluded from the operation of a loop, and for which we cannot write a test around that specific aspect? My instinct is to say no.

Your trivial example may be an example of lack of what I referred to as test-drivenness in my blog, writing about NinjaTurtles. Meaning in the case that you have not refactored this method as far as you should.