Why do I have a .NET Core API performance issue using IHttpClientFactory when ramping up users?

Question

I have a .NET Core WebAPI my React UI is calling to Authenticate Users. It calls a 3rd Party API for this. Everything works fine but we have started to do Performance testing on it and it is not scaling well when we ramp up the Users attempting to log on concurrently. (taking 30 secs)

The 3rd Party API we call are saying they are responding in milliseconds.

My API is hosted in Kubertnetes container on AWS. I have added AWS X-ray to the code to try and get further information though I am not really sure on how to interpret the results.

The code is quite straightforward - This is a snippet from MyAuthenticationProvider class (the constructor takes a metric collector (for AWS X-Ray and and securityProvider http client for making the call)

        metricCollector.StartCollection("Stage 1");
        HttpResponseMessage response = await securityProvider.SendAsync(requestMessage);
        metricCollector.EndCollection();

The X-Ray image for the above code is:

Is X-Ray showing that it is indeed waiting 30+ Seconds for this API to return a response and I should reach out to that company for further investigation on there side even though they are telling me all traffic is getting responded too in milli-seconds.

Or could it be how I have defined the http client used in MyAuthProvider class in Startup.cs that is not scaling correctly when the concurrent users ramps up?

This is the code for that in Startup.cs

      services.AddTransient<IMyAuthenticationProvider>(ctx =>
        {
            IHttpClientFactory clientFactory = ctx.GetRequiredService<IHttpClientFactory>();
            return new MyAuthenticationProvider(
                clientFactory.CreateClient("3RDPARTYAUTHCLIENT"),
                ctx.GetService<IMetricCollector>());
        });

Another thing I was thing to improve performance is introducing Redis to cache some of these responses as they are getting calling multiple times for different operations but the result will be the same

Answer 1

While you're only creating 1 named HttpClient, you've set the service lifetime of IMyAuthenticationProvider to transient.

This means that essentially you're losing out on most of the benefits of a single HttpClient by creating a new instance of IMyAuthenticationProvider every time something requests for one (which in the best-case scenario, will be synonymous with every client request but not to be mistaken with scoped services).

This can massively slow down your application & may be the cause of the badly performing scaling of the application.

You're trying to clearly use a single HttpClient, which would typically be static or wrapped as a non-static instance inside a singleton class. and is still a good solution for short-lived console applications etc. however in this case I'd allow IHttpClientFactory to resolve the client.

The primary goal of IHttpClientFactory in ASP.NET Core is to ensure that HttpClient instances are created appropriately (taking into account things like DNS changes which a single HttpClient instance won't take care of) while at the same time eliminating socket exhaustion.

Injected HttpClient instances by IHttpClientFactory have a transient lifetime (documentation is conflicting & mentions transient 2x & scoped 1x for some absurd reason) and so I'd set the lifetime of IMyAuthenticationProvider to scoped to allow it to be reused as much as possible.

Having a longer running singleton IHttpClientFactory, in this case, with an injected shorter-lived scoped HttpClient should not be done.

MSFT:

Do not resolve a scoped service from a singleton and be careful not to do so indirectly

While the injected HttpClient object is transient, using the IHttpClientFactory enables pooling of HttpMessageHandler objects that can and will be reused by multiple HttpClient instances.

Try:

services.AddHttpClient<IMyAuthenticationProvider, MyAuthenticationProvider>();
services.AddHttpClient<IMetricCollector, MetricCollector>();
...

services.AddScoped<IMyAuthenticationProvider, MyAuthenticationProvider>();

public class MyAuthenticationProvider : IMyAuthenticationProvider
{
  private readonly HttpClient _httpClient;

  public MyAuthenticationProvider(HttpClient httpClient)
  {
      _httpClient = httpClient;
  }

  ...
}