HttpClient not performing under heavy load

Question

I have serious problems with the .NET HttpClient under heavy load. I have created a GitHub repo to demonstrate the problem. Would greatly appreciate any help. Yes, I am only using one HttpClient per authority. Same problem with http and https. Not a server-side problem, as server maintains responsive the whole time.

https://github.com/erikbra/HttpClientProblems/issues/1

Typically get the following error. System.Net trace log says it's calling ::Abort on the connection. But very difficult to get any more info.

System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.WinHttpException: The connection with the server was terminated abnormally
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Threading.Tasks.RendezvousAwaitable`1.GetResult()
   at System.Net.Http.WinHttpHandler.<StartRequest>d__105.MoveNext()
   --- End of inner exception stack trace ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at System.Net.Http.HttpClient.<FinishSendAsyncBuffered>d__58.MoveNext()
   --- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at UnitTests.Call_Api.<CallApi>d__22.MoveNext() in C:\Users\ErikBrandstadmoen\Source\Repos\HttpClientProblems\UnitTests.NetCore\Call_Api.cs:line 204
   --- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at UnitTests.Call_Api.<Measure>d__23.MoveNext() in C:\Users\ErikBrandstadmoen\Source\Repos\HttpClientProblems\UnitTests.NetCore\Call_Api.cs:line 219

[UPDATE]

To be precise, I want to understand what is happening. Obviously the connection is being cut of, but why? The server is still responding, so it's not a limitation on the server side. Where is the bottleneck?

What happens if I e.g. set ServicePointManager.DefaultConnectionLimit to 2, and start 4 tasks? Won't they be served in turn? Similar with, say, 128 and 1000. But somewhere it stops scaling. How can I investigate further what the bottleneck is? I only get a TaskCanceledException, and those carry no more detail. Any tips?

Answer 1

I didn't go into code details of your github solution but this smells as an expected behavior. There is a setting of maximum connection numbers per ServicePoint and its normal that you run out of connections quicker on longer running requests when you do parallel, async execution.

This might be a good starting point: https://docs.microsoft.com/en-us/dotnet/framework/network-programming/managing-connections

There is no definite, unified answer about how many connections is ok but you should keep your MaxDegreeOfParalelism below the number of your maximum connections allowed and you're safe.

[UPDATE]

Also, what you might want to take a look at is System.Net.Http.WinHttpHandler:

https://msdn.microsoft.com/en-us/library/system.net.http.winhttphandler(v=vs.105).aspx

When you make an instance of HttpClient, you can do something like this:

var httpMessageHandler = new System.Net.Http.WinHttpHandler();
//here you set different handler options (see docs link above)
var httpClient = new HttpClient(httpMessageHandler);