Multiple HTTP requests trigger HTTP Client timeout

Question

I have an app that's sending 500 HTTP requests asynchronously. All requests processed after 15 seconds fail because of the timeout on the HTTP Client, even when the requested endpoint already returned a 200 OK.

The code is very straight-forward. Here we take a chunk of requests (500), and execute them asynchronously. It should be noted that the function below is an Azure function running on the consumption-based plan.

    public async Task RunBatch(List<Request> requests)
    {
        if (requests != null && requests .Count > 0)
        {
            var tasks = new Task[requests.Count];
            var i = 0;
            foreach (var request in requests)
            {
                var request = new HttpRequestMessage(HttpMethod.Post, new Uri(request.Url));
                request.Content = new StringContent(request.BodyString, Encoding.UTF8, "application/json");
                tasks[i] = _httpClient.SendAsync(request);
                i++;
            }

            await Task.WhenAll(tasks);
        }
    }

The following code exists in my constructor

_httpClient = new HttpClient();
_httpClient.Timeout = new TimeSpan(0, 0, 15); // 15 seconds

Here are logs from Azure.

I'd like each request to have a timeout of 15 seconds. However, I need it to give me an accurate response code whenever my server gets around to processing the awaited request. Is this possible?

I should note: with a higher Http timeout (1 min), all requests succeed.

Answer 1

Personally, I think attempting to issue 500 concurrent requests is always going to be error prone. You mention that you're doing it asynchronously, but in reality there's not a whole lot of asynchrony in your code as you fire-up 500 "hot" tasks then wait for them all to finish.

I would use a semaphore to control how many requests can be made at once. You may have to play with the numbers to find the sweet spot.

The following code works well in LINQPad (although bing quickly notices the odd number of requests and starts adding a CAPTCHA to the page):

// using System.Threading;
async Task Main()
{
    var httpClient = new HttpClient();
    var urls = Enumerable.Range(1, 500).Select(e => "https://www.bing.com/").ToList();
    
    // 10 concurrent requests - tweak this number
    var semaphore = new SemaphoreSlim(10, 10);
    
    var tasks = urls.Select(u => MakeRequest(u, semaphore, httpClient));
        
    var allResponses = await Task.WhenAll(tasks);
    
    // Do something with allResponses
}

private async Task<string> MakeRequest(string url, SemaphoreSlim semaphore, HttpClient httpClient)
{
    try
    {
        await semaphore.WaitAsync();
        var request = new HttpRequestMessage(HttpMethod.Get, new Uri(url));
        var response = await httpClient.SendAsync(request);
        
        // Add an optional delay for further throttling:
        //await Task.Delay(TimeSpan.FromMilliseconds(100));
        
        return await response.Content.ReadAsStringAsync();
    }
    finally
    {
        semaphore.Release();
    }
}