C# How to simplify unit testing string parameters using AutoFixture

Question

I'm trying to create a simple way to test string parameters in unit tests, for most string parameters, I want to check the behaviour when the parameter is Null, Empty or consists only of whitespaces.

In most of the cases, I'm checking the parameters with string.IsNullOrWhiteSpace(), and throw an exception if it has one of those three values.

Now for unit testing, it seems I have to write, for every string parameter, three unit tests. One for null values, one for empty values and one for Whitespaces.

Imagine a method with 3 or 4 string parameters, then I need to write 9 or 12 unit tests...

Can anyone come up with a simple way to test this? Maybe using AutoFixture?

Answer 1

To avoid duplicating the same test multiple times you can write a parameterized test.

If you're using xUnit, you would write a so called theory. A theory means you're proving a principle, that is that a certain function behaves as expected when given different samples of the same kind of input data. For example:

[Theory]
[InlineData(null)]
[InlineData("")]
[InlineData(" ")]
public void Should_throw_argument_exception_when_input_string_is_invalid(string input)
{
    Assert.Throws<ArgumentException>(() => SystemUnderTest.SomeFunction(input));
}

The xUnit runner will run this test multiple times, each time assigning the input parameter to one of the values specified in the [InlineData] attribute.

If the function being tested has more than one parameter, you might not care what values are passed to the remaining ones, as long as at least one of them is a string that's either null, empty or contains only whitespace.

In this case, you can combine parameterized tests with AutoFixture. AutoFixture is designed to give you general-purpose test data that is good enough to use in most scenarios when you don't necessarily care about what the exact values are.

In order to use it with xUnit theories, you'll need to add the AutoFixture.Xunit NuGet package (or AutoFixture.Xunit2 depending on which version you're using) to your test project and use the InlineAutoData attribute:

[Theory]
[InlineAutoData(null)]
[InlineAutoData("")]
[InlineAutoData(" ")]
public void Should_throw_argument_exception_when_the_first_input_string_is_invalid(
    string input1,
    string input2,
    string input3)
{
    Assert.Throws<ArgumentException>(() =>
        SystemUnderTest.SomeFunction(input1, input2, input3));
}

Only the input1 parameter will have a specific value, while the rest will be assigned random strings by AutoFixture.

One thing to note here is that the values passed through the [InlineAutoData] attribute are assigned to the test parameters based on their position. Since we need to test the same behavior for all three parameters individually, we'll have to write three tests:

[Theory]
[InlineAutoData(null)]
[InlineAutoData("")]
[InlineAutoData(" ")]
public void Should_throw_argument_exception_when_the_second_input_string_is_invalid(
    string input2,
    string input1,
    string input3)
{
    Assert.Throws<ArgumentException>(() =>
        SystemUnderTest.SomeFunction(input1, input2, input3));
}

[Theory]
[InlineAutoData(null)]
[InlineAutoData("")]
[InlineAutoData(" ")]
public void Should_throw_argument_exception_when_the_third_input_string_is_invalid(
    string input3,
    string input1,
    string input2)
{
    Assert.Throws<ArgumentException>(() =>
        SystemUnderTest.SomeFunction(input1, input2, input3));
}

A word on property-based testing

I can't help but think that these kinds of test scenarios are a perfect match for property-based testing. Without going into too much detail, property-based testing is about proving a specific behavior (or "property") of a function by running it multiple times with generated input.

In other words:

Property-based tests make statements about the output of your code based on the input, and these statements are verified for many different possible inputs.

In your case, you could write a single test to verify that the function throws an ArgumentException whenever at least one of the arguments is either null, an empty string or a string that only contains whitespace.

In .NET, you can use a library called FsCheck to write and execute property-based tests. While the API is primarily designed to be used from F#, it can also be used from C#.

However, in this particular scenario, I think you're better off sticking with regular parameterized tests and AutoFixture to achieve the same goal. In fact, writing these tests with FsCheck and C# would end up being more verbose and wouldn't really buy you much in terms of robustness.

Update: AutoFixture.Idioms

As @RubenBartelink pointed out in the comments, there is another option. AutoFixture encapsulates common assertions in a small library called AutoFixture.Idioms. You can use it to centralize the expectations on how the string parameters are validated by a method and use them in your tests.

While I do have my reservations on this approach, I'll add it here as another possible solution for the sake of completeness:

[Theory, AutoData]
public void Should_throw_argument_exception_when_the_input_strings_are_invalid(
    ValidatesTheStringArguments assertion)
{
    var sut = typeof(SystemUnderTest).GetMethod("SomeMethod");

    assertion.Verify(sut);
}

public class ValidatesTheStringArguments : GuardClauseAssertion
{
    public ValidatesTheStringArguments(ISpecimenBuilder builder)
        : base(
              builder,
              new CompositeBehaviorExpectation(
                  new NullReferenceBehaviorExpectation(),
                  new EmptyStringBehaviorExpectation(),
                  new WhitespaceOnlyStringBehaviorExpectation()))
    {
    }
}

public class EmptyStringBehaviorExpectation : IBehaviorExpectation
{
    public void Verify(IGuardClauseCommand command)
    {
        if (!command.RequestedType.IsClass
            && !command.RequestedType.IsInterface)
        {
            return;
        }

        try
        {
            command.Execute(string.Empty);
        }
        catch (ArgumentException)
        {
            return;
        }
        catch (Exception e)
        {
            throw command.CreateException("empty", e);
        }

        throw command.CreateException("empty");
    }
}

public class WhitespaceOnlyStringBehaviorExpectation : IBehaviorExpectation
{
    public void Verify(IGuardClauseCommand command)
    {
        if (!command.RequestedType.IsClass
            && !command.RequestedType.IsInterface)
        {
            return;
        }

        try
        {
            command.Execute(" ");
        }
        catch (ArgumentException)
        {
            return;
        }
        catch (Exception e)
        {
            throw command.CreateException("whitespace", e);
        }

        throw command.CreateException("whitespace");
    }
}

Based on the expectations expressed in NullReferenceBehaviorExpectation, EmptyStringBehaviorExpectation, and WhitespaceOnlyStringBehaviorExpectation AutoFixture will automatically attempt to invoke the method named "SomeMethod" with null, empty strings and whitespace respectively.

If the method doesn't throw the correct exception – as specified in the catch block inside the expectation classes – then AutoFixture will itself throw an exception explaining what happend. Here's an example:

An attempt was made to assign the value whitespace to the parameter "p1" of the method "SomeMethod", and no Guard Clause prevented this. Are you missing a Guard Clause?

You can also use AutoFixture.Idioms without parameterized tests by simply instantiating the objects yourself:

[Fact]
public void Should_throw_argument_exception_when_the_input_strings_are_invalid()
{
    var assertion = new ValidatesTheStringArguments(new Fixture());
    var sut = typeof(SystemUnderTest).GetMethod("SomeMethod");

    assertion.Verify(sut);
}