HttpClient query occasionally hangs

Question

I initialise HttpClient like so:

public static CookieContainer cookieContainer = new CookieContainer();
public static HttpClient httpClient = new HttpClient(new HttpClientHandler() { AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate, CookieContainer = cookieContainer }) { Timeout = TimeSpan.FromSeconds(120) };

so all queries should throw TaskCanceledException if no response is received within 120 seconds. But some queries (like 1 of 100 000-1 000 000) hang infinitely.

I wrote following code:

public static async Task<HttpResponse> DownloadAsync2(HttpRequestMessage httpRequestMessage)
{
    HttpResponse response = new HttpResponse { Success = false, StatusCode = (int)HttpStatusCode.RequestTimeout, Response = "Timeout????????" };
    Task task;
    if (await Task.WhenAny(
        task = Task.Run(async () =>
        {
            try
            {
                HttpResponseMessage r = await Global.httpClient.SendAsync(httpRequestMessage).ConfigureAwait(false);
                response = new HttpResponse { Success = true, StatusCode = (int)r.StatusCode, Response = await r.Content.ReadAsStringAsync().ConfigureAwait(false) };
            }
            catch (TaskCanceledException)
            {
                response = new HttpResponse { Success = false, StatusCode = (int)HttpStatusCode.RequestTimeout, Response = "Timeout" };
            }
            catch (Exception ex)
            {
                response = new HttpResponse { Success = false, StatusCode = -1, Response = ex.Message + ": " + ex.InnerException };
            }
        }),
        Task.Run(async () =>
        {
            await Task.Delay(TimeSpan.FromSeconds(150)).ConfigureAwait(false);
        })
    ).ConfigureAwait(false) != task)
    {
        Log("150 seconds passed");
    }
    return response;
}

which actually occasionally executes Log("150 seconds passed");.

I call it like so:

HttpResponse r = await DownloadAsync2(new HttpRequestMessage
{
    RequestUri = new Uri("https://address.com"),
    Method = HttpMethod.Get
}).ConfigureAwait(false);

Why TaskCanceledException sometimes isn't thrown after 120 seconds?

Answer 1

I don't know with how frequency you call DownloadAsync2, but your code smells a lot for bursting and starving ThreadPool.

Initial number of threads in ThreadPool by default is limited to number of CPU logical cores (usually 12 for today normal systems) and in case of unavailability of threads in ThreadPool, 500ms takes for each new thread to be generated.

So for example:

for (int i = 0; i < 1000; i++)
{
    HttpResponse r = await DownloadAsync2(new HttpRequestMessage
    {
        RequestUri = new Uri("https://address.com"),
        Method = HttpMethod.Get
    }).ConfigureAwait(false);
}

This code with a high chance will be freezed, specially if you have some lock or any cpu intensive tasks somewhere in your code. Because you invoke new thread per calling DownloadAsync2 so all threads of ThreadPool consumed and many more of them still needed.

I know maybe you say "all of my tasks have been awaited and they release for other works". but they also consumed for starting new DownloadAsync2 threads and you will reach the point that after finishing await Global.httpClient.SendAsync no thread remains for re-assigning and completing the task.

So method have to wait until one thread being available or generated to complete (even after timeout). Rare but feasible.