Does XUnit share fixture instances across test classes?

Question

This is a simplified version of some code I have:

public class FixtureData
{
    public object SomeValue { get; set; }
}

public class TestForNull : IClassFixture<FixtureData>
{
    private readonly FixtureData _data;

    public TestForNull(FixtureData data)
    {
        _data = data;
    }

    [Fact]
    public void TestForNull()
    {
        _data.SomeValue = null;
        Assert.Null(_data.SomeValue);
    }
}

public class TestForObject : IClassFixture<FixtureData>
{
    private readonly FixtureData _data;

    public TestForObject(FixtureData data)
    {
        _data = data;
    }

    [Fact]
    public void TestForObject()
    {
        Assert.NotNull(_data.SomeValue);
    }
}

Both classes are not marked with any collection attribute. They both belong to the same assembly.

I am seeing these test fail (but only sporadically), which can only be explained by XUnit sharing the FixtureData instance across test classes, and TestForNull running first (since it has side effects).

However, the XUnit documentation makes it clear that class fixtures are to "shared object instance across tests in a single class".

Is this a bug? Should I change the way in which I'm using the fixtures?

I'm using xUnit for .NET Core 2.3.1.

Answer 1

As Ruben Bartelink says in his response, "SELECT is not broken", meaning that this is a very core feature that XUnit, a well proven test framework and it's very very unlikely that the issue is on their side.

Furthermore, digging into XUnit code, this is what it does to generate class fixtures: (Source)

var createClassFixtureAsyncTasks = new List<Task>();
foreach (var interfaceType in testClassTypeInfo.ImplementedInterfaces.Where(i => i.GetTypeInfo().IsGenericType && i.GetGenericTypeDefinition() == typeof(IClassFixture<>)))
    createClassFixtureAsyncTasks.Add(CreateClassFixtureAsync(interfaceType.GetTypeInfo().GenericTypeArguments.Single()));

if (TestClass.TestCollection.CollectionDefinition != null)
{
    var declarationType = ((IReflectionTypeInfo)TestClass.TestCollection.CollectionDefinition).Type;
    foreach (var interfaceType in declarationType.GetTypeInfo().ImplementedInterfaces.Where(i => i.GetTypeInfo().IsGenericType && i.GetGenericTypeDefinition() == typeof(IClassFixture<>)))
        createClassFixtureAsyncTasks.Add(CreateClassFixtureAsync(interfaceType.GetTypeInfo().GenericTypeArguments.Single()));
}

await Task.WhenAll(createClassFixtureAsyncTasks);

From the calls to CreateClassFixtureAsync it's easy to tell that class fixtures are regenerated each time for a test case.

Then why the behavior I observed?

I accidentally simplified more in my example that I should have. I found that this might be a better example of what was going on:

public class FixtureData
{
    public object SomeValue => HiddenSingleton.Instance.SomeValue;
}

public class HiddenSingleton
{
    private static HiddenSingleton _instance;
    public static HiddenSingleton Instance
    {
        get
        {
            if (_instance != null) return _instance;

            _instance = new HiddenSingleton();
            return _instance;
        }
    }

    public object SomeValue { get; set; }
}

public class TestForNull : IClassFixture<FixtureData>
{
    private readonly FixtureData _data;

    public TestForNull(FixtureData data)
    {
        _data = data;
    }

    [Fact]
    public void TestForNull()
    {
        _data.SomeValue = null;
        Assert.Null(_data.SomeValue);
    }
}

public class TestForObject : IClassFixture<FixtureData>
{
    private readonly FixtureData _data;

    public TestForObject(FixtureData data)
    {
        _data = data;
    }

    [Fact]
    public void TestForObject()
    {
        Assert.NotNull(_data.SomeValue);
    }
}

In this case, looking at it directly, it is quite obvious: even though XUnit generates an independent FixtureData instance for each test, the singleton actually makes it so that they use the same instance.

In my case, I was looking at the test classes independently and I did not realize there was a singleton, so I assumed the problem was related with test fixtures (incorrect assumption). And because of the parts I missed when asking the question, it was impossible for someone to correctly pick up on what was wrong.

Moral of the story:

• Trust tested frameworks (Specially if they specialize in testing!)
• Stop overusing singletons
• Make sure you've got all relevant parts of the problem before asking in StackOverflow

Answer 2

No, you've read the docs correctly. TL;DR because Test Classes can run in parallel, the Class Fixtures have to be independent, that's the whole point of them.

I'd be surprised if there's a bug in xUnit wrt this as this feature/facility is stable and not undergoing change.

If you can make your actual sample fail, then, yes that's a bug in xUnit but I'm saying a) it's not failing now b) you're not going to make it fail c) SELECT is not broken ;)

Hope this helps ;)