Thread pool differences between .NET Framework and .NET Core, Thread Pool starvation

Question

I stumbled upon an unexpected behavior when trying to pass the working code from .Net Framework 4.6.1 to .Net Core 3.1

This is a simplification of the code:

static  void Main(string[] args)
{
    for (int i = 0; i < 20; i++)
    {
        ThreadPool.QueueUserWorkItem(o =>
        {
            Console.Write($"In, ");
            RestClient restClient = new RestClient($"http://google.com");
            RestRequest restRequest = new RestRequest();
            var response = restClient.Get(restRequest);

            Console.Write($"Out, ");
        });
    }

    Console.ReadLine();
}

The expected output on the console is a list of "In" followed by mixed "In" & "Out" and finally some "Out" as a result of a multithreaded work. and this works as expected on .Net Framework. Something like this:

In, In, In, In, In, In, In, In, In, In, In, In, In, In, In, Out, In, Out,
In, Out, In, Out, In, Out, In, Out, Out, Out, Out, Out, Out, Out, Out,
Out, Out, Out, Out, Out, Out, Out,

But when running the exact same code on .Net Core 3.1 (same machine) it looks like we get back to write the "out" line only after all the "in" thread finished (I tested this with a lot more than 20).

In, In, In, In, In, In, In, In, In, In, In, In, In, In, In, In, In, In,
In, In, Out, Out, Out, Out, Out, Out, Out, Out, Out, Out, Out, Out, Out,
Out, Out, Out, Out, Out, Out, Out,

Meaning there is starvation on the process and if the number of added work items to the thread pool is infinite (e.g. depends on API) the HTTP response will never be handled.

I assume this happens because of the way the ThreadPool algorithm chooses the next thread to handle this is a nice article on the subject

What I don't understand is why it doesn't happen on .Net Framework, and if I can make it work somehow on .Net Core.

P.s. I'm not trying to avoid working with TPL I'm just trying to get to the bottom of this.

Any suggestions?

Answer 1

[EDITED] Here's what I found

The difference between .NET Core and .NET Framework is in the implementation of HttpWebRequest.GetResponse(). In .NET Framework, it uses Thread.SpinWait(1) and in .NET Core, it does SendRequest().GetAwaiter().GetResult() - essentially calling the async implementation and doing a Wait() on it.

Async method calls rely on TaskScheduler to execute continuations. The TaskScheduler relies on the ThreadPool.

Normally, the thread pool starts with minThreads = # cores. Then it uses some algorithm to slowly increase the number of threads until it reaches maxThreads.

The code immediately posts 20 blocking jobs to the the thread pool. Continuation jobs are queued after them. The thread pool slowly increases the number of threads to accommodate the download jobs, and only then it adds a thread which processes the first Continuation job.

Another interesting twist is that if you set both min and max threads to the same low number and run the code, it deadlocks. That's because the the Continuation will never receive a thread to execute on. Some more info about the deadlock here.

There are multiple ways to solve this

1. Avoid mixing sync and async code. Just go async all the way (if you can)
2. Use ThreadPool.SetMinThreads to start with a sufficient number of threads. You need at least the number of threads as the expected number of concurrent download jobs.
3. In the sample code, if you add even a 10-50ms delay between posting the download jobs, the continuation jobs have a chance to get scheduled in between.

(the question uses something called RestClient which is probably using HttpClient or HttpWebRequest under the hood. The code below uses HttpWebRequest)

private static void Main(string[] args)
{
    //ThreadPool.SetMinThreads(4, 4);
    //ThreadPool.SetMaxThreads(4, 4);
    for (var i = 0; i < 20; i++)
        ThreadPool.QueueUserWorkItem(o =>
        {
            Console.Write("In, ");

            var r = (HttpWebRequest) WebRequest.Create("http://google.com");
            r.GetResponse();
            //Try this in .Net Framework and get the same result in as in .NET Core.
            //That's because in .NET Core r.GetResponse() essentially does r.GetResponseAsync().Wait()
            //r.GetResponseAsync().Wait();  

            Console.Write("Out, ");
        });

    Console.ReadLine();
}