Usage of Assert.Inconclusive

Question

Inconclusive test is a test for which you cannot determine the result. For example, what if you had a test that uses some kind of an external resource (Internet connection, for example). If the connection is currently not available, this doesn't really mean the test is a failure. On the other hand, you shouldn't just mark it as successful without actually running it through. So you mark it as inconclusive and this can be seen in the test report.

NOTE: In general, you shouldn't use such external resources in your tests, since this can make the tests fragile.

For tests that haven't been finished yet, I use MbUnit's Explicit attribute.

When I use VS to generate the unit tests, I got Assert.Inclusive for generated test methods and usually I change the assertion so something else when I work them. I use the question marks of Assert.Inconclusive in test result as markers to quickly tell me which tests I have not yet completed.

Well, it's just the way I use it. From its name "Inconclusive", I guess you can use to indicate your undeterministic state as long as you document what it means.

However, from the description of your Average() method, I think that maybe your unit test is not atomic enough to cover just one "unit", one specific scenario. Sometimes, I write 2 or 3 unit test methods for a single method. Or you can break your Average() method to smaller methods covering single responsibilities. That way, you can unit test those smaller methods before unit testing you Average() one.

---

Johannes,

This is how I would implement the Sum() and Average() methods.

public static class MyMath
{
    private static void ValidateInput(ICollection<int> numbers)
    {
        if (numbers == null)
            throw new ArgumentNullException("numbers", "Null input.  Nothing to compute!");
        if (numbers.Count == 0)
            throw new ArgumentException("Input is empty.  Nothing to compute!");
    }

    public static int Sum(int[] numbers)
    {
        ValidateInput(numbers);

        var total = 0;
        foreach (var number in numbers)
            total += number;

        return total;
    }

    public static double Average(int[] numbers)
    {
        ValidateInput(numbers);
        return Sum(numbers) / numbers.Length;
    }
}

For simplicity, I just throw ArgumentException exceptions from the ValidateInput(ICollection<int>) method. You can also check for possibility to overflow and throw OverflowException in the ValidateInput(ICollection<int>) method.

With that said, here's how I would test the Average(int[]) function.

[TestMethod]
public void AverageTest_GoodInput()
{
    int[] numbers = {1, 2, 3};
    const double expected = 2.0;
    var actual = MyMath.Average(numbers);
    Assert.AreEqual(expected, actual);
}

[TestMethod]
[ExpectedException(typeof(ArgumentNullException))]
public void AverageTest_NullInput()
{
    int[] numbers = null;
    MyMath.Average(numbers);
}

[TestMethod]
[ExpectedException(typeof(ArgumentException))]
public void AverageTest_EmptyInput()
{
    var numbers = new int[0];
    MyMath.Average(numbers);
}

With these tests setup, I can be certain that when all the tests pass, my function is correct. Well, except for the case of overflow. Now I can go back to the ValidateInput(ICollection<int>) method to add logic to check for overflow, then add one more test to expect the OverflowException to be thrown for the kind of inputs that cause overflow. Or doing it in the opposite order if you like to approach using TDD.

I hope this helps clarify the idea.

I only Assert.Inconclusive on unit tests I haven't written yet. Sometimes when writing something I realize some corner case that I don't want to miss. So I quickly jump over, write the test method with a descriptive name and add a single line of Assert.Inconclusive.

The reason for doing this is that it provides documentation on things that I need to test without interrupting my work flow too much. It also makes it possible to filter out test failures quickly in the result list. Having an inconclusive failure means I haven't broken anything I just more tests to write.

Assert.Inconclusive indicates that either:

I have not yet written the test;I've only created the test method

-or-

My test has a dependency and that dependency is not available. For example

List<Customer> custs = o.GetAllCustomers();
if (custs.Count == 0)
{
  Assert.Inconclusive("No customers to test");
  return;
}