Remove failure message from multiple assertion failure in NUnit

Question

I have configured my tests to retry up to x number of times in the event of a failure, to ensure the failure is legitimate and not a fluke during the run. I do not log the error message on the initial failure.

However, I am noticing that if I am running a test, the first test fails, and then the second test passes and I check for any assertion failures via TestContext.CurrentContext.Result.Message and notice the first iteration failure is logged and my test is shown as failed, even though the test passed during the second iteration. If both tests fail, I will receive a "Multiple failures or warnings in test."

I would like to retain the final run's failure only vs all the failures for all the iterations. Is there a way to remove the initial failure from the TestContext.CurrentCOntext.Result.Message?

Edit: I am using NUnit v 3.10.1 and when I downgraded to v.3.4.0 I got the experience I desired without any modification to my code.

Answer 1

Use NUnit's [Retry(5)] attribute on your test to retry the test if it fails. Workarounds like in the link you posted depend on the undocumented internal behavior of NUnit that may change between releases.

Update based on your comment below, if you need to handle unexpected exceptions, wrap the flaky code that might throw in a try/catch block, then do your assertions outside of that block.

[Test]
[Retry(5)]
public void TestFlakyMethod()
{
    int result = 0;
    try
    {
        result = FlakyAdd(2, 2);
    }
    catch(Exception ex)
    {
        Assert.Fail($"Test failed with unexpected exception, {ex.Message}");
    }
    Assert.That(result, Is.EqualTo(4));
}

int FlakyAdd(int x, int y)
{
    var rand = new Random();
    if (rand.NextDouble() > 0.5)
        throw new ArgumentOutOfRangeException();

    return x + y;
}

Adding to the above, you can also use Assert.DoesNotThrow, it is a bit cleaner and easier to write.

[Test]
[Retry(5)]
public void TestFlakyMethod()
{
    int result = 0;
    Assert.DoesNotThrow(() => {{
        result = FlakyAdd(2, 2);
    });
    Assert.That(result, Is.EqualTo(4));
}