IHttpHandler versus HttpTaskAsyncHandler performance

Question

We have a webapp that routes many requests through a .NET IHttpHandler (called proxy.ashx) for CORS and security purposes. Some resources load fast, others load slow based on the large amount of computation required for those resources. This is expected.

During heavy load, proxy.ashx slows to a crawl, and ALL resources take forever to load. During these peak load times, if you bypass the proxy and load the resource directly, it loads immediately which means that the proxy is the bottleneck.
(i.e. http://server/proxy.ashx?url=http://some_resource loads slow, but http://some_resource loads fast on its own).

I had a hypothesis that the reduced responsiveness was because the IHttpHandler was coded synchronously, and when too many long-running requests are active, the IIS request threads are all busy. I created a quick A/B testing app to verify my hypothesis, and my test results are showing that this is not the case.

This article is where I am basing understanding of the request thread pool.

On the Web server, the .NET Framework maintains a pool of threads that are used to service ASP.NET requests. When a request arrives, a thread from the pool is dispatched to process that request. If the request is processed synchronously, the thread that processes the request is blocked while the request is being processed, and that thread cannot service another request. ...
However, during an asynchronous call, the server is not blocked from responding to other requests while it waits for the first request to complete. Therefore, asynchronous requests prevent request queuing when there are many requests that invoke long-running operations.

In my example below, in theory, the synchronous handler should hog request threads after a certain threshold, preventing more new requests from starting. The async handler should allow MANY more requests to queue up, because every request almost immediately yields its request thread back to the thread pool while it awaits Task.Delay, allowing that request thread to process a new request while the previous request is still awaiting.

Synchronous HttpHandler

<%@ WebHandler Language="C#" Class="SyncHandler" %>
using System.Web;
using System.Threading;
public class SyncHandler : IHttpHandler
{
    public void ProcessRequest(HttpContext context)
    {
        //BLOCKING artifical pause to simulate network activity
        Thread.Sleep(300);
        var Response = context.Response;
        Response.Write("sync response");
    }
    public bool IsReusable { get { return true; } }
}

Asynchronous Handler

<%@ WebHandler Language="C#" Class="AsyncHandler" %>
using System.Web;
using System.Threading.Tasks;

public class AsyncHandler : HttpTaskAsyncHandler
{
    public override async Task ProcessRequestAsync(HttpContext context)
    {
        //NON-BLOCKING artificial pause to simulate network activity
        await Task.Delay(300);
        var Response = context.Response;
        Response.Write("async response");
    }
    public override bool IsReusable { get { return true; } }
}

Benchmarking

I ran some benchmarks using the apache benchmark utility. Here's the command I'm using (changing the numbers for the results below, obviously).

ab -n 1000 -c 10 http://localhost/AsyncProxyTest/Sync.ashx
ab -n 1000 -c 10 http://localhost/AsyncProxyTest/Async.ashx

Results

1,000 requests, 10 at a time

• sync: 30.10 requests/second
• async: 32.05 request/second

10,000 requests, 100 at a time

• sync: 33.02 requests/second
• async: 32.05 requests/second

10,000 requests, 1,000 at a time

• sync: 32.55 requests/second
• async: 32.05 requests/second

As you can see, sync versus async seems to have almost no effect (at least not enough to make it worth the switch).

My question is: Did I mess something up in my tests that is not accurately modeling this concept?

Answer 1

There is a limitation in the desktop version of IIS that limits concurrent requests to 10 at a time (see this post). This limitation is not present in IIS Express and not present in IIS on Windows server.

There is nothing wrong with the tests, they just need to be run on an unrestricted web server. I re-ran these tests using IIS on Windows Server and my findings were exactly as I expected from my initial hypothesis.

Here are the results again. (Note: these results are from a very active development server, so there will probably be some fluxuation)

1,000 requests, 10 at a time
(These results are the same because we are not going above the request limit)

• sync: 31.88 requests/second
• async: 29.13 request/second

10,000 requests, 100 at a time

• sync: 68.53 requests/second
• async: 310.66 requests/second

10,000 requests, 1,000 at a time

• sync: 55.09 requests/second
• async: 669.41 requests/second

Another metric that I captured was the maximum number of concurrent requests running. This is how I discovered the limit of 10 for my local machine. After running the tests again on the Windows server, the max for sync was ~48 concurrent requests. For async, it was 301, meaning that async/await definitely yields a higher throughput when dealing with nonblocking calls.