Specflow - GenerateFeatureFileCodeBehindTask has failed unexpectedly

Question

I have a problem with SpecFlow. We are using Azure Devops and when I build solution on my local machine it works perfectly but during Azure Devops build I am getting following error:

[error]C:\Windows\ServiceProfiles\NetworkService\.nuget\packages\specflow.tools.msbuild.generation\3.1.86\build\SpecFlow.Tools.MsBuild.Generation.targets(93,5): Error MSB4018: The "GenerateFeatureFileCodeBehindTask" task failed unexpectedly.
System.IO.FileNotFoundException: Could not load file or assembly 'TechTalk.SpecFlow, Version=3.1.0.0, Culture=neutral, PublicKeyToken=0778194805d6db41'. The system cannot find the file specified.
File name: 'TechTalk.SpecFlow, Version=3.1.0.0, Culture=neutral, PublicKeyToken=0778194805d6db41'
   at System.ModuleHandle.ResolveTypeHandleInternal(RuntimeModule module, Int32 typeToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext)
   at System.Reflection.RuntimeModule.ResolveType(Int32 metadataToken, Type[] genericTypeArguments, Type[] genericMethodArguments)
   at System.Reflection.CustomAttribute.FilterCustomAttributeRecord(MetadataToken caCtorToken, MetadataImport& scope, RuntimeModule decoratedModule, MetadataToken decoratedToken, RuntimeType attributeFilterType, Boolean mustBeInheritable, ListBuilder`1& derivedAttributes, RuntimeType& attributeType, IRuntimeMethodInfo& ctor, Boolean& ctorHasParameters, Boolean& isVarArg)
   at System.Reflection.CustomAttribute.AddCustomAttributes(ListBuilder`1& attributes, RuntimeModule decoratedModule, Int32 decoratedMetadataToken, RuntimeType attributeFilterType, Boolean mustBeInheritable, ListBuilder`1 derivedAttributes)
   at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeModule decoratedModule, Int32 decoratedMetadataToken, Int32 pcaCount, RuntimeType attributeFilterType)
   at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeAssembly assembly, RuntimeType caType)
   at System.Reflection.RuntimeAssembly.GetCustomAttributes(Type attributeType, Boolean inherit)
   at System.Attribute.GetCustomAttributes(Assembly element, Type attributeType, Boolean inherit)
   at System.Attribute.GetCustomAttribute(Assembly element, Type attributeType, Boolean inherit)
   at System.Attribute.GetCustomAttribute(Assembly element, Type attributeType)
   at TechTalk.SpecFlow.Generator.Plugins.GeneratorPluginLoader.LoadPlugin(PluginDescriptor pluginDescriptor)
   at TechTalk.SpecFlow.Generator.GeneratorContainerBuilder.LoadPlugins(ObjectContainer container, GeneratorPluginEvents generatorPluginEvents, UnitTestProviderConfiguration unitTestProviderConfiguration, IEnumerable`1 generatorPlugins)
   at TechTalk.SpecFlow.Generator.GeneratorContainerBuilder.CreateContainer(SpecFlowConfigurationHolder configurationHolder, ProjectSettings projectSettings, IEnumerable`1 generatorPluginInfos, IObjectContainer parentObjectContainer)
   at SpecFlow.Tools.MsBuild.Generation.WrappedGeneratorContainerBuilder.BuildGeneratorContainer(SpecFlowConfigurationHolder specFlowConfigurationHolder, ProjectSettings projectSettings, IReadOnlyCollection`1 generatorPluginInfos, IObjectContainer rootObjectContainer)
   at SpecFlow.Tools.MsBuild.Generation.GenerateFeatureFileCodeBehindTaskExecutor.Execute()
   at SpecFlow.Tools.MsBuild.Generation.GenerateFeatureFileCodeBehindTask.Execute()
   at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
   at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask)

        <PackageReference Include="FluentAssertions" Version="5.10.2" />
        <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.5.0" />
        <PackageReference Include="MSTest.TestAdapter" Version="2.1.0" />
        <PackageReference Include="MSTest.TestFramework" Version="2.1.0" />
        <PackageReference Include="NUnit" Version="3.12.0" />
        <PackageReference Include="Selenium.Support" Version="3.141.0" />
        <PackageReference Include="Selenium.WebDriver" Version="3.141.0" />
        <PackageReference Include="SpecFlow" Version="3.1.86" />
        <PackageReference Include="SpecFlow.MsTest" Version="3.1.86" />
        <PackageReference Include="SpecFlow.Tools.MsBuild.Generation" Version="3.1.86" />

I have tried using it with nUnit, MsTest, xUnit and it alway fails. Target framework: .NET Core 3.1. Build agent: vs2019-win2019

Answer 1

We've encountered same problem, after longer digging I found that there is an issue created for this in their git repo https://github.com/SpecFlowOSS/SpecFlow/issues/1912. Quoting from git issue:

MSBuild 16.5 has a new assembly loading approach for tasks that may have side effects. (which has been an ask for years, but it behaves a bit differently when loading dependencies) You can try to set MSBUILDSINGLELOADCONTEXT=1 (env var) to get the old behavior.

Try to set it in your build pipeline as:

 variables:
    ...
    MSBUILDSINGLELOADCONTEXT: '1'
    ...

This helped in our case.

---

EDIT: This issue seems to be fixed in newer version of SpecFlow (3.3.15+).

Answer 2

Had the same problem, turned out that it was version issues between .net core SDK and Specflow. We downgraded .net core sdk to 3.1.102 and that helped

Answer 3

Had the same problem for SpecFlow 3.4.8 after migration from SpecFlow 2.

Since After version SpecFlow 3.3.30 don’t need to add the SpecFlow.Tools.MSBuild.Generation package anymore

so i have removed SpecFlow.Tools.MSBuild.Generation package, however issue still there.

After that i have added specflow.json to test project and issue gone