How do I remove the delay between HTTP Requests when using Asynchronous actions in ASP.NET?

Question

I am using HttpClient to send a GET request to a server inside of a while loop

while (cycle < maxcycle)
{

    var searchParameters = new ASearchParameters
    {
        Page = cycle++,
        id = getid
    };

    var searchResponse = await Client.SearchAsync(searchParameters);

}

and the SearchAsync contains

public async Task<AuctionResponse> SearchAsync()
{
    
    var uriString = "Contains a https url with parameters"
    var searchResponseMessage = await HttpClient.GetAsync(uriString);

    return await Deserialize<AuctionResponse>(searchResponseMessage);

    
}

The thing is after every request there is a delay before the next request is started. you can see this in fiddler timeline and also in fiddler there is "Tunnel To" example.com:443 before every request

Question : Why is there a delay and how to remove it ?

Answer 1

I see two things that are happening here. First, depending on the deserializer, it may take a while to translate your response back into an object. You might want to time that step and see if that's not the majority of your time spent. Second, the SSL handshake (the origin of your "tunnel to") does require a round trip to establish the SSL channel. I thought HttpClient sent a Keep-Alive header by default, but you may want to see if it is A) not being sent or B) being rejected. If you are re-establishing an SSL channel for each request, that could easily take on the order of a hundred ms all by itself (depending upon the server/network load).

If you're using Fiddler, you can enable the ability to inspect SSL traffic to see what the actual request/response headers are.