Unexpected outcome of node.js vs ASP.NET Core performance test

Question

As many others have alluded, the comparison lacks context.
At the time of its release, the async approach of node.js was revolutionary. Since then other languages and web frameworks have been adopting the approaches they took mainstream.

To understand what the difference meant, you need to simulate a blocking request that represents some IO workload, such as a database request. In a thread-per-request system, this will exhaust the threadpool and new requests will be put in to a queue waiting for an available thread.
With non-blocking-io frameworks this does not happen.

Consider this node.js server that waits 1 second before responding

const server = http.createServer((req, res) => {
  setTimeout(() => {
    res.statusCode = 200;
    res.end();
  }, 1000);
});

Now let's throw 100 concurrent conenctions at it, for 10s. So we expect about 1000 requests to complete.

$ wrk -t100 -c100 -d10s http://localhost:8000
Running 10s test @ http://localhost:8000
  100 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.01s    10.14ms   1.16s    99.57%
    Req/Sec     0.13      0.34     1.00     86.77%
  922 requests in 10.09s, 89.14KB read
Requests/sec:     91.34
Transfer/sec:      8.83KB

As you can see we get in the ballpark with 922 completed.

Now consider the following asp.net code, written as though async/await were not supported yet, therefore dating us back to the node.js launch era.

app.Run((context) =>
{
    Thread.Sleep(1000);
    context.Response.StatusCode = 200;
    return Task.CompletedTask;
});

$ wrk -t100 -c100 -d10s http://localhost:5000
Running 10s test @ http://localhost:5000
  100 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.08s    74.62ms   1.15s   100.00%
    Req/Sec     0.00      0.00     0.00    100.00%
  62 requests in 10.07s, 5.57KB read
  Socket errors: connect 0, read 0, write 0, timeout 54
Requests/sec:      6.16
Transfer/sec:     566.51B

62! Here we see the limit of the threadpool. By tuning it up we could get more concurrent requests happening, but at the cost of more server resources.

For these IO-bound workloads, the move to avoid blocking the processing threads was that dramatic.

Now let's bring it to today, where that influence has rippled through the industry and allow dotnet to take advantage of its improvements.

app.Run(async (context) =>
{
    await Task.Delay(1000);
    context.Response.StatusCode = 200;
});

$ wrk -t100 -c100 -d10s http://localhost:5000
Running 10s test @ http://localhost:5000
  100 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.01s    19.84ms   1.16s    98.26%
    Req/Sec     0.12      0.32     1.00     88.06%
  921 requests in 10.09s, 82.75KB read
Requests/sec:     91.28
Transfer/sec:      8.20KB

No surprises here, we now match node.js.

So what does all this mean?

Your impressions that node.js is the "fastest" come from an era we are no longer living in. Add to that it was never node/js/v8 that were "fast", it was that they broke the thread-per-request model. Everyone else has been catching up.

If your goal is the fastest possible processing of single requests, then look at the serious benchmarks instead of rolling your own. But if instead what you want is simply something that scales to modern standards, then go for whichever language you like and make sure you are not blocking those threads.

Disclaimer: All code written, and tests run, on an ageing MacBook Air during a sleepy Sunday morning. Feel free to grab the code and try it on Windows or tweak to your needs - https://github.com/csainty/nodejs-vs-aspnetcore

Node Frameworks like Express and Koa have a terrible overhead. "Raw" Node is significantly faster.

I haven't tried it, but there's a newer framework that gets very close to "Raw" Node performance: https://github.com/aerojs/aero

(see benchmark on that page)

update: Here are some figures: https://github.com/blitzprog/webserver-benchmarks

Node:
    31336.78
    31940.29
Aero:
    29922.20
    27738.14
Restify:
    19403.99
    19744.61
Express:
    19020.79
    18937.67
Koa:
    16182.02
    16631.97
Koala:
    5806.04
    6111.47
Hapi:
    497.56
    500.00

As you can see, the overheads in the most popular node.js frameworks are VERY significant!