Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Specflow Contextual Assertions

We use Specflow and currently, most of our assertions are done using Fluent Assertions, ie look something like:
myval.Should().NotBe(null)

All is working well, however there are some cases in which we want the same code to sometimes Assert Inconclusive, and sometimes not.

For example, let us assume we have a step which calls a method to install the AUT.
The test which asserts that the AUT can be installed correctly will use this method as part of a WHEN/THEN step.
In this instance, if installation fails, we want to proceed as normal and fail the test.
GIVEN The AUT is NOT installed WHEN I install the AUT

However, for all other tests, the same method may be called as part of a GIVEN step or as part of a BeforeScenario / BeforeFeature hook, and in that instance, if the installation fails, the test should be failed as inconclusive, because it was not the test itself that failed, but the Setup stage. GIVEN the AUT is installed WHEN I perform function X of the AUT

So in this example, let us assume that the step definitions for WHEN I install the AUT in the first test, and GIVEN the AUT is installed in the second test simply call a helper method with something like AppFacade.Install()

Of course we can litter AppFacade.Install() and all our other helper methods with conditionals etc, but I was wondering if anyone out there has a more elegant solution - ie way to extend an assertion engine so that it automatically applies this logic - I see no barriers on the Specflow side to achieving this insofar as you can examine the Specflow context objects to detect what kind of step you are on.
Ideally we would like to stick with Fluent Assertions, but if this overly complicates things, we would be willing to reconsider.

like image 715
Clive Galway Avatar asked Sep 18 '25 14:09

Clive Galway


1 Answers

I was going to suggest you were not asking the right question, since you are implying that when a Specflow step execution fails the test should be inconclusive, but you already realized it.

So to answer the actual question you have:

Is it possible to throw an exception which can mark the test as inconclusive given the above logic?

Yes, you can use the after AfterStep hook (see Specflow hooks here) to check the is there was any exception thrown and execute Assert.Inconclusive() if you're using Nunit or any equivalent method, like Specflow's ScenarioContext.Current.Pending() to not be attached to the testing framework installed, the only caveat is that ScenarioContext.Current.Pending() does not clear ScenarioContext.TestStatus, so your test runner will mark it as red and you'll have to use reflection to set the TestStatus property as described here.

It would look something like this:

[AfterStep]
public void Check()
{
    if (ScenarioContext.Current.TestError is AssertionException &&
        ScenarioContext.Current.CurrentScenarioBlock == ScenarioBlock.Given)
        Assert.Inconclusive(ScenarioContext.Current.TestError.Message);
}

Edit

Here is a demo I created.

like image 104
Maximo Dominguez Avatar answered Sep 20 '25 03:09

Maximo Dominguez