IHost.RunAsync() Never Returns

Question

I'm building a .NET Core 3.1 app that will run a BackgroundService in a Docker container. While I've implemented the startup and shutdown tasks for the BackgroundService and the service is definitely shutting down when triggered via SIGTERM, I'm finding that the await host.RunAsync() call never completes - meaning the remaining code in my Main() block isn't executed.

Am I missing something or should I not expect the RunAsync() call to return control after the background service has completed stopping?

(Updated with the simplest repro I can come up with...)

    using System;
    using System.Threading;
    using System.Threading.Tasks;
    using Microsoft.Extensions.DependencyInjection;
    using Microsoft.Extensions.Hosting;

    namespace BackgroundServiceTest
    {
        class Program
        {
            static async Task Main(string[] args)
            {
                Console.WriteLine("Main: starting");
                try
                {
                    using var host = CreateHostBuilder(args).Build();

                    Console.WriteLine("Main: Waiting for RunAsync to complete");

                    await host.RunAsync();

                    Console.WriteLine("Main: RunAsync has completed");
                }
                finally
                {
                    Console.WriteLine("Main: stopping");
                }
            }

            public static IHostBuilder CreateHostBuilder(string[] args) =>
                Host.CreateDefaultBuilder(args)
                    .UseConsoleLifetime()
                    .ConfigureServices((hostContext, services) =>
                    {
                        services.AddHostedService<Worker>();

                        // give the service 120 seconds to shut down gracefully before whacking it forcefully
                        services.Configure<HostOptions>(options => options.ShutdownTimeout = TimeSpan.FromSeconds(120));
                    });

        }

        class Worker : BackgroundService
        {
            protected override async Task ExecuteAsync(CancellationToken stoppingToken)
            {
                Console.WriteLine("Worker: ExecuteAsync called...");
                try
                {
                    while (!stoppingToken.IsCancellationRequested)
                    {
                        await Task.Delay(TimeSpan.FromSeconds(10), stoppingToken);
                        Console.WriteLine("Worker: ExecuteAsync is still running...");
                    }
                }
                catch (OperationCanceledException) // will get thrown if TaskDelay() gets cancelled by stoppingToken
                {
                    Console.WriteLine("Worker: OperationCanceledException caught...");
                }
                finally
                {
                    Console.WriteLine("Worker: ExecuteAsync is terminating...");
                }
            }

            public override Task StartAsync(CancellationToken cancellationToken)
            {
                Console.WriteLine("Worker: StartAsync called...");
                return base.StartAsync(cancellationToken);
            }

            public override async Task StopAsync(CancellationToken cancellationToken)
            {
                Console.WriteLine("Worker: StopAsync called...");
                await base.StopAsync(cancellationToken);
            }

            public override void Dispose()
            {
                Console.WriteLine("Worker: Dispose called...");
                base.Dispose();
            }
        }
    }

Dockerfile:

    #See https://aka.ms/containerfastmode to understand how Visual Studio uses this Dockerfile to build your images for faster debugging.

    FROM mcr.microsoft.com/dotnet/core/runtime:3.1-buster-slim AS base
    WORKDIR /app

    FROM mcr.microsoft.com/dotnet/core/sdk:3.1-buster AS build
    WORKDIR /src
    COPY ["BackgroundServiceTest.csproj", "./"]
    RUN dotnet restore "BackgroundServiceTest.csproj"
    COPY . .
    WORKDIR "/src/"
    RUN dotnet build "BackgroundServiceTest.csproj" -c Release -o /app/build

    FROM build AS publish
    RUN dotnet publish "BackgroundServiceTest.csproj" -c Release -o /app/publish

    FROM base AS final
    WORKDIR /app
    COPY --from=publish /app/publish .
    ENTRYPOINT ["dotnet", "BackgroundServiceTest.dll"]

docker-compose.yml:

    version: '3.4'

    services:
      backgroundservicetest:
        image: ${DOCKER_REGISTRY-}backgroundservicetest
        build:
          context: .
          dockerfile: Dockerfile

Run this via docker-compose up --build and then in a second window, run docker stop -t 90 backgroundservicetest_backgroundservicetest_1

Console output shows that the Worker shuts down and gets disposed, but the application (apparently) terminates before RunAsync() returns.

    Successfully built 3aa605d4798f
    Successfully tagged backgroundservicetest:latest
    Recreating backgroundservicetest_backgroundservicetest_1 ... done
    Attaching to backgroundservicetest_backgroundservicetest_1
    backgroundservicetest_1  | Main: starting
    backgroundservicetest_1  | Main: Waiting for RunAsync to complete
    backgroundservicetest_1  | Worker: StartAsync called...
    backgroundservicetest_1  | Worker: ExecuteAsync called...
    backgroundservicetest_1  | info: Microsoft.Hosting.Lifetime[0]
    backgroundservicetest_1  |       Application started. Press Ctrl+C to shut down.
    backgroundservicetest_1  | info: Microsoft.Hosting.Lifetime[0]
    backgroundservicetest_1  |       Hosting environment: Production
    backgroundservicetest_1  | info: Microsoft.Hosting.Lifetime[0]
    backgroundservicetest_1  |       Content root path: /app
    backgroundservicetest_1  | Worker: ExecuteAsync is still running...
    backgroundservicetest_1  | Worker: ExecuteAsync is still running...
    backgroundservicetest_1  | info: Microsoft.Hosting.Lifetime[0]
    backgroundservicetest_1  |       Application is shutting down...
    backgroundservicetest_1  | Worker: StopAsync called...
    backgroundservicetest_1  | Worker: OperationCanceledException caught...
    backgroundservicetest_1  | Worker: ExecuteAsync is terminating...
    backgroundservicetest_1  | Worker: Dispose called...
    backgroundservicetest_backgroundservicetest_1 exited with code 0

Answer 1

After a lengthy discussion on Github, it turns out that some minor refactoring solves the problem. In a nutshell, .RunAsync() blocks until the host completes and disposes the host instance, which (apparently) terminates the application.

By changing the code to call .StartAsync() and then await host.WaitForShutdownAsync(), control does return back to Main() as expected. The last step is to dispose the host in a finally block as shown:

static async Task Main(string[] args)
{
    Console.WriteLine("Main: starting");
    IHost host = null;
    try
    {
        host = CreateHostBuilder(args).Build();

        Console.WriteLine("Main: Waiting for RunAsync to complete");
        await host.StartAsync();

        await host.WaitForShutdownAsync();

        Console.WriteLine("Main: RunAsync has completed");
    }
    finally
    {
        Console.WriteLine("Main: stopping");

        if (host is IAsyncDisposable d) await d.DisposeAsync();
    }
}