MSTest unit test passes by itself, fails when other tests are run

Question

I'm having trouble with some MSTest unit tests that pass when I run them individually but fail when I run the entire unit test class. The tests test some code that SLaks helped me with earlier, and he warned me what I was doing wasn't thread-safe. However, now my code is more complicated and I don't know how to go about making it thread-safe. Here's what I have:

public static class DLLConfig
{
    private static string _domain;

    public static string Domain
    {
        get
        {
            return _domain = AlwaysReadFromFile
                ? readCredentialFromFile(DOMAIN_TAG)
                : _domain ?? readCredentialFromFile(DOMAIN_TAG);
        }
    }
}

And my test is simple:

string expected = "the value I know exists in the file";
string actual = DLLConfig.Domain;
Assert.AreEqual(expected, actual);

When I run this test by itself, it passes. When I run it alongside all the other tests in the test class (which perform similar checks on different properties), actual is null and the test fails. I note this is not a problem with a property whose type is a custom Enum type; maybe I'm having this problem with the Domain property because it is a string? Or maybe it's a multi-threaded issue with how MSTest works?

Answer 1

I suspect the other tests are modifying some value in the DLLConfig class that is causing the result of the getter to change. Unit tests should always be run from a known initial state, so you should either set this up in the test method itself, or in a method marked with the TestInitialize attribute which is run before each test.

Answer 2

Your test is depending on an external file. Instead of calling a function that directly accesses the file you should have DLLConfig.Domain call a method in another class.

public static string Domain
{
    get
    {
        return _domain = AlwaysReadFromFile
            ? CredentialReader.Read(DOMAIN_TAG)
            : _domain ?? CredentialReader.Read(DOMAIN_TAG);
    }
}

Then you can initialize DllConfig with a mock/fake/stub CredentialReader where you can control its return value. Remember you are testing that DLLConfig.Domain is returning the correct value based on the AlwaysReadFromFile condition. You shouldn't be testing where that value is coming from (or if it even exists) at the same time.

Making your DLLConfig class more "testable" has the added benefit of separating out concerns. When you are thinking about a class and can't help but say "This class does this AND that" (abstracting configuration data and reading that data from a file) its a good bet the class is mixing concerns and trying to do to much. If DLLConfig is an abstraction of configuration data it should focus only on that and leave where the data comes from to another class.

Answer 3

If none of the answers above worked for you, i solved this issue by adding Thread.Sleep(1) before the assertion in the failing test...

Looks like tests synchronization is missed somewhere... Please note that my tests were not order dependant, that i haven't any static member nor external dependency.