Is async HttpClient from .Net 4.5 a bad choice for intensive load applications?

Question

Besides the tests mentioned in the question, I recently created some new ones involving much fewer HTTP calls (5000 compared to 1 million previously) but on requests that took much longer to execute (500 milliseconds compared to around 1 millisecond previously). Both tester applications, the synchronously multithreaded one (based on HttpWebRequest) and asynchronous I/O one (based on HTTP client) produced similar results: about 10 seconds to execute using around 3% of the CPU and 30 MB of memory. The only difference between the two testers was that the multithreaded one used 310 threads to execute, while the asynchronous one just 22. So in an application that would have combined both I/O bound and CPU bound operations the asynchronous version would have produced better results because there would have been more CPU time available for the threads performing CPU operations, which are the ones that actually need it (threads waiting for I/O operations to complete are just wasting).

As a conclusion to my tests, asynchronous HTTP calls are not the best option when dealing with very fast requests. The reason behind that is that when running a task that contains an asynchronous I/O call, the thread on which the task is started is quit as soon the as the asynchronous call is made and the rest of the task is registered as a callback. Then, when the I/O operation completes, the callback is queued for execution on the first available thread. All this creates an overhead, which makes fast I/O operations to be more efficient when executed on the thread that started them.

Asynchronous HTTP calls are a good option when dealing with long or potentially long I/O operations because it doesn't keep any threads busy on waiting for the I/O operations to complete. This decreases the overall number of threads used by an application allowing more CPU time to be spent by CPU bound operations. Furthermore, on applications that only allocate a limited number of threads (like it is the case with web applications), asynchronous I/O prevents thread pool thread depletion, which can happen if performing I/O calls synchronously.

So, async HttpClient is not a bottleneck for intensive load applications. It is just that by its nature it is not very well suited for very fast HTTP requests, instead it is ideal for long or potentially long ones, especially inside applications that only have a limited number of threads available. Also, it is a good practice to limit concurrency via ServicePointManager.DefaultConnectionLimit with a value that high enough to ensure a good level of parallelism, but low enough to prevent ephemeral port depletion. You can find more details on the tests and conclusions presented for this question here.

One thing to consider that might be affecting your results is that with the HttpWebRequest you are not getting the ResponseStream and consuming that stream. With HttpClient, by default it will copy the network stream into a memory stream. In order to use HttpClient in the same way that you are currently using HttpWebRquest you would need to do

var requestMessage = new HttpRequestMessage() {RequestUri = URL};
Task<HttpResponseMessage> getTask = httpClient.SendAsync(requestMessage, HttpCompletionOption.ResponseHeadersRead);

The other thing is that I'm not really sure what the real difference, from a threading perspective, you are actually testing. If you dig into the depths of HttpClientHandler it simply does Task.Factory.StartNew in order to perform an async request. The threading behaviour is delegated to the synchronization context in exactly the same way as your example with HttpWebRequest example is done.

Undoubtedly, HttpClient add some overhead as by default it uses HttpWebRequest as its transport library. So you will always be able to get better perf with a HttpWebRequest directly whilst using HttpClientHandler. The benefits that HttpClient brings is with the standard classes like HttpResponseMessage, HttpRequestMessage, HttpContent and all the strongly typed headers. In itself it is not a perf optimization.

While this does not directly answer the 'async' part of the OP's question, this addresses an error in the implementation he is using.

If you want your application to scale, avoid using instance-based HttpClients. The difference is HUGE! Depending on the load, you will see very different performance numbers. The HttpClient was designed to be re-used across requests. This was confirmed by guys on the BCL team who wrote it.

A recent project I had was to help a very large and well-known online computer retailer scale out for Black Friday/holiday traffic for some new systems. We ran into some performance issues around the usage of HttpClient. Since it implements IDisposable, the devs did what you would normally do by creating an instance and placing it inside of a using() statement. Once we started load testing the app brought the server to its knees - yes, the server not just the app. The reason is that every instance of HttpClient opens an I/O Completion Port on the server. Because of non-deterministic finalization of GC and the fact that you are working with computer resources that span across multiple OSI layers, closing network ports can take a while. In fact Windows OS itself can take up to 20 secs to close a port (per Microsoft). We were opening ports faster than they could be closed - server port exhaustion which hammered the CPU to 100%. My fix was to change the HttpClient to a static instance which solved the problem. Yes, it is a disposable resource, but any overhead is vastly outweighed by the difference in performance. I encourage you to do some load testing to see how your app behaves.

Also answered at link below:

What is the overhead of creating a new HttpClient per call in a WebAPI client?

https://www.asp.net/web-api/overview/advanced/calling-a-web-api-from-a-net-client