Static HttpClient still creating TIME_WAIT tcp ports

Question

I am experiencing some interesting behavior with the HttpClient from the .NET Framework (4.5.1+, 4.6.1 and 4.7.2). I have proposed some changes in a project at work to not dispose of the HttpClient on each use because of the known issue with high TCP port usage, see https://aspnetmonsters.com/2016/08/2016-08-27-httpclientwrong/.

I have investigated the changes to check that things were working as expected and found that we are still experiencing the same TIME_WAIT ports as before.

To confirm that my proposed changes were correct I have added some extra tracing to the application that confirm that I am using the same instance of the HttpClient through out the application. I have since used simple test application (taken from the aspnetmonsters site linked above.

using System;
using System.Net.Http;

namespace ConsoleApplication
{
    public class Program
    {
        private static HttpClientHandler { UseDefaultCredentials = true };
        private static HttpClient Client = new HttpClient(handler);
        public static async Task Main(string[] args) 
        {
            Console.WriteLine("Starting connections");
            for(int i = 0; i<10; i++)
            {
                var result = await Client.GetAsync("http://localhost:51000");
                Console.WriteLine(result.StatusCode);
            }
            Console.WriteLine("Connections done");
            Console.ReadLine();
        }
    }
}

The issue only occurs when connecting to a site that is hosted in IIS using Windows Authentication. I can reproduce the issue easily by setting the Authentication to Anonymous (problem goes away) and back to Windows Authentication (problem reoccurs).

The issue with Windows Authentication does not seem to be limited to the scope of the provider. It has the same issue if you us Negotiate or NTLM. Also the issue occurs if the machine is just a workstation or part of a domain.

Out of interest I created a dotnet core 2.1.0 console app and the issue is not present at all and works as expected.

TLDR: Does any one have any idea on how to fix this, or is it likely to be a bug?

Answer 1

Short version

Use .NET Core 2.1 if you want to reuse connections with NTLM authentication

Long version

I was quite surprised to see that the "old" HttpClient does use a different connection for each request when NTLM authentication is used. This isn't a bug - before .NET Core 2.1 HttpClient would use HttpWebRequest which closes the connection after every NTLM authenticated call.

This is described in the documentation of the HttpWebRequest.UnsafeAuthenticatedConnectionSharing property which can be used to enable sharing of the connection :

The default value for this property is false, which causes the current connection to be closed after a request is completed. Your application must go through the authentication sequence every time it issues a new request.

If this property is set to true, the connection used to retrieve the response remains open after the authentication has been performed. In this case, other requests that have this property set to true may use the connection without re-authenticating.

The risk is that :

If a connection has been authenticated for user A, user B may reuse A's connection; user B's request is fulfilled based on the credentials of user A.

If one understands the risks, and the application doesn't use impersonation, one could configure HttpClient with a WebRequestHandler and set the UnsafeAuthenticatedConnectionSharing, eg :

HttpClient _client;

public void InitTheClient()
{
    var handler=new WebRequestHandler
                { 
                    UseDefaultCredentials=true,
                    UnsafeAuthenticatedConnectionSharing =true
                };
    _client=new HttpClient(handler); 
}

WebRequestHandler doesn't expose the HttpWebRequest.ConnectionGroupName that would allow to group connections eg by ID, so it can't handle impersonation.

.NET Core 2.1

HttpClient was rewritten in .NET Core 2.1 and implements all the HTTP, networking functionality using sockets, minimal allocations, connection pooling etc. It also handles the NTLM challenge/response flow separatelly so the same socket connection can be used to serve different authenticated requests.

If anyone is interested, you can chase the calls from HttpClient to SocketsHttpHanlder to HttpConnectionPoolManager , HttpConnectionPool, HttpConnection, AuthenticationHelper.NtAuth and then back to HttpConnection to send the raw bytes.