Why does this xunit test deadlock (on a single cpu VM)?

Question

Running the following test on a single CPU VM (Ubuntu 18.4)

using System;
using System.Threading;
using System.Threading.Tasks;
using Xunit;

public class AsyncStuffTest
{
    [Fact]
    public void AsyncTest()
    {
        SomethingAsync().Wait();
    }

    private static async Task SomethingAsync()
    {
        Console.WriteLine("before async loop...");
        await Task.Factory.StartNew(() => {
                                        for (int i = 0; i < 10; i++)
                                        {
                                            Console.WriteLine("in async loop...");
                                            Thread.Sleep(500);     
                                        }
                                    });
        Console.WriteLine("after async loop...");
    }
}

results in this:

Build started, please wait...
Build completed.

Test run for /home/agent/fancypants/bin/Debug/netcoreapp2.1/fancypants.dll(.NETCoreApp,Version=v2.1)
Microsoft (R) Test Execution Command Line Tool Version 15.7.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
before async loop...
in async loop...
in async loop...
in async loop...
in async loop...
in async loop...
in async loop...
in async loop...
in async loop...
in async loop...
in async loop...

The process seems to deadlock and never proceeds to the expected output after async loop...

Running on my dev machine everything works fine.

Note: I know about the possibility of async testing in xunit. This is more or less a question of interest. Especially since this problem only affects xunit, a console application is terminating fine:

~/fancypants2$ dotnet run
before async loop...
in async loop...
in async loop...
in async loop...
in async loop...
in async loop...
in async loop...
in async loop...
in async loop...
in async loop...
in async loop...
after async loop...
~/fancypants2$

Update: Read about some recent fixes related to async in xunit, so I tried this with 2.4.0-beta.2.build4010, but there is no change.

Answer 1

After two days of wrapping my head around SynchronizationContext (basically the best information without talking too much about "UI Thread" can be found here: https://blogs.msdn.microsoft.com/pfxteam/2012/01/20/await-synchronizationcontext-and-console-apps/ ) I understand what's going on.

The console application does not provide any SynchronizationContext, so the CLR will offload the task to a thread on the thread pool. There are enough threads available, no matter what CPU the machine has. Everything works fine.

xunit indeed provides a Xunit.Sdk.MaxConcurrencySyncContext that actively manages the amount of running threads. The maximum level of concurrency defaults to the amount of logical CPUs you have, however, it can be configured. The thread running the test is already maxing out this limitation, so the task completion is blocking.

All of this was a try to reproduce an issue with a much more complex ASP.Net Core Web application, that behaves strange on the mentioned single CPU build agent. An integration test uses a collection wide shared fixture that starts up a TestServer

public class ServiceHostFixture : IAsyncLifetime
{
    public async Task InitializeAsync()
    {
        IWebHostBuilder host = new WebHostBuilder()
                    .UseEnvironment("Production")
                    .UseStartup<Startup>();

        Server = new TestServer(host);
    }

    public async Task DisposeAsync()
    {
        Server.Dispose();
    }
}

While there is an interesting bit in Startup.Configure(IApplicationBuilder app):

app.ApplicationServices
    .GetRequiredService<IApplicationLifetime>()
    .ApplicationStarted
    .Register(async () => {
                    try
                    {
                        // it blocks here in xunit
                        await EnsureSomeBasicStuffExistenceInTheDatabaseAsync();
                    }
                    catch (Exception ex)
                    {
                        Logger.Fatal(ex, "Application could not be started");
                    }
                });

On my (8 logical CPU) machine, it works fine, on a single cpu web host it works fine, but xunit on a single cpu deadlocks. If you read carefully the documentation of CancellationToken what ApplicationStartedactually is, you'll find this:

The current System.Threading.ExecutionContext, if one exists, will be captured along with the delegate and will be used when executing it.

Combining this with the difference between ASP.Net Core and xunit reveals the issue. What I did was the following workaround:

app.ApplicationServices
    .GetRequiredService<IApplicationLifetime>()
    .ApplicationStarted
    .Register(async () => {
                    try
                    {
                        if (SynchronizationContext.Current == null)
                        {
                            // normal ASP.Net Core environment does not have a synchronization context, 
                            // no problem with await here, it will be executed on the thread pool
                            await EnsureSomeBasicStuffExistenceInTheDatabaseAsync;
                        }
                        else
                        {
                            // xunit uses it's own SynchronizationContext that allows a maximum thread count
                            // equal to the logical cpu count (that is 1 on our single cpu build agents). So
                            // when we're trying to await something here, the task get's scheduled to xunit's 
                            // synchronization context, which is already at it's limit running the test thread
                            // so we end up in a deadlock here.
                            // solution is to run the await explicitly on the thread pool by using Task.Run
                            Task.Run(() => EnsureSomeBasicStuffExistenceInTheDatabaseAsync()).Wait();
                        }
                    }
                    catch (Exception ex)
                    {
                        Logger.Fatal(ex, "Application could not be started");
                    }
                });