Howto resolve .net test hangs on "Starting test execution, please wait..."

Question

I am trying to execute dotnet test from the command line on our Jenkins build server, but it just hangs on:

Starting test execution, please wait...

It works fine when running this command locally

If I switch to using dotnet xunit it fails with the following:

15:42:57 Locating binaries for framework netcoreapp2.1...
15:42:58 Running .NET Core 2.1 tests for framework netcoreapp2.1...
15:42:58 The specified framework version '2.1' could not be parsed
15:42:58 The specified framework 'Microsoft.NETCore.App', version '2.1' was not found.
15:42:58   - Check application dependencies and target a framework version installed at:
15:42:58       C:\Program Files\dotnet\
15:42:58   - Installing .NET Core prerequisites might help resolve this problem:
15:42:58       http://go.microsoft.com/fwlink/?LinkID=798306&clcid=0x409
15:42:58   - The .NET Core framework and SDK can be installed from:
15:42:58       https://aka.ms/dotnet-download
15:42:58   - The following versions are installed:
15:42:58       2.0.6 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
15:42:58       2.0.9 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
15:42:58       2.1.2 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
15:42:58       2.1.3 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
15:42:58       2.1.4 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]

As per the error message, we have installed the dotnet core SDK on the server but we seem to be missing something.

My test project looks like:

<Project Sdk="Microsoft.NET.Sdk">    
  <PropertyGroup>
    <TargetFramework>netcoreapp2.1</TargetFramework>    
    <IsPackable>false</IsPackable>
  </PropertyGroup>    
  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.App" Version="2.1.1" />
    <PackageReference Include="Microsoft.AspNetCore.Mvc.Core" Version="2.1.1" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.7.0" />
    <PackageReference Include="Moq" Version="4.10.0" />
    <PackageReference Include="RichardSzalay.MockHttp" Version="5.0.0" />
    <PackageReference Include="xunit" Version="2.3.1" />
    <PackageReference Include="xunit.runner.console" Version="2.4.0" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.3.1" />
    <DotNetCliToolReference Include="dotnet-xunit" Version="2.3.1" />
  </ItemGroup>        
</Project>

Answer 1

The versioning was a red herring and it turned out to be much simpler problem. My tests were testing Async controller methods, and my non-async tests were doing:

var result = controller.PostAsync(request).Result;

as soon as I changed the tests themselves to use the async/await pattern they worked fine:

var result = await controller.PostAsync(request);

Something that helped me diagnose the issue was using the dotnet test --verbosity d argument. When using this it was outputting some tests that were passing rather than just "Starting test execution, please wait". Interestingly every time I run the command it would execute a different number of tests before appearing to get stuck. This suggested there was perhaps some sort of thread deadlock issue which led me to the solution. I'm still unsure why the command ran fine on my local machine but not our Jenkins agent.

Answer 2

The solution that worked for me was to add an AssemblyInfo.cs file with the following contents:

using Xunit;

[assembly: CollectionBehavior(DisableTestParallelization = true)]

Once I did this, the tests started running in my CI builds again (in my case, Azure Pipelines).

Answer 3

Sometimes xUnit tests hang forever because the test runner pipeline can't handle parallel threads.

To check if that's the issue try adding xunit.runner.json file to the tests project root

  <ItemGroup>
    <None Update="xunit.runner.json">
      <CopyToOutputDirectory>Always</CopyToOutputDirectory>
    </None>
  </ItemGroup>

And put inside 1 thread config.

{
  "parallelizeTestCollections": true,
  "maxParallelThreads": -1
}

Answer 4

My problem was an async method inside a constructor:

MyAsyncMethod().GetAwaiter().GetResult();

I fixed the problem by moving the async method out of the constructor and implementing xunits IAsyncLifetime interface (Await Tasks in Test Setup Code in xUnit.net?)

public async Task InitializeAsync()
{
    await MyAsyncMethod();
}