Why is async considered better performing than multithreading? [closed]

Question

I understand both asynchronous and multithreaded programming and I have done both and can do them with ease. However one thing still bugs me: why is the general consensus that async is better performing than multithreading? (added: I'm talking about the case where either approach is viable and you get to make a choice)

At the first glance the reason seems clear - less threads, less work for the OS scheduler, less memory wasted for stack space. But... I don't feel like these arguments hold water. Let's look at them individually:

1. Less work for the OS scheduler. True, but does it mean less work in total? There are still N tasks running in parallel, SOMEBODY has to switch between them. Seems to me like we've simply taken the work from the OS kernel and started doing it in our own userland code. But the amount of work that needs to be done hasn't changed because of it. Where then does the efficiency come from?
2. Less memory wasted for stack space. Or is it?
   • First of all, I don't know about other OS'es, but at least in Windows the stack space for a thread isn't committed all at once. There are virtual memory addresses reserved, but actual memory is committed only when it's needed.
   • And even if it was committed, it wouldn't matter much, because just allocating memory doesn't slow your program down. Not unless you run out of it, and modern computers have enough memory for thousands of stacks, especially servers.
   • And even if the stacks DO get committed and DO end up causing a memory shortage, most stacks will only be used a bit at the start (if your program is flirting with a stack overflow, you've got bigger problems to worry about). Which means that it will be possible to page most of them out anyway.
   • The real problem with large memory usage is that the CPU cache gets trashed a lot. When you've got lots of data all over the place that you need, and the CPU cache cannot keep up with it all and needs to fetch things from main RAM again and again - that's when things get slow. But async programming doesn't help in this in any way. If anything, it actively uses more memory. Instead of a lean stack frame we now have separate Task objects allocated on the heap essentially for every stack frame which contain state and local variables and callback references and everything. Plus it's fragmented all over the address space which gives even more headaches to the CPU cache, because pre-fetching will be useless.

So... which elephant in the room have I missed?

Answer 1

why is the general consensus that async is better performing than multithreading? (added: I'm talking about the case where either approach is viable and you get to make a choice)

On the server side, async lets you make maximum use of threads. Why have one thread handle a single connection when it can handle hundreds? On the server side, it's not an "async vs threads" scenario - it's an "async and threads" scenario.

On the client side - where either approach is truly viable - it doesn't matter as much. So what if you spin up an extra unnecessary thread? It's just not that big of a deal, even for mobile apps these days. While technically, async can help be more efficient especially in a memory- and battery-constrained device, at this point in history it's not that terribly important. However, even on the client side, async has a tremendous benefit in that it allows you to write serial code rather than mucking around with callbacks.

There are still N tasks running in parallel, SOMEBODY has to switch between them.

No. I/O tasks as used by async do not "run" anywhere, and do not need to be "switched" to. On Windows, I/O tasks use IOCPs underneath, and I/O tasks do not "run" - they only "complete", which happens as the result of a system interrupt. More info in my blog post "There Is No Thread".

Where then does the efficiency come from?

The word "efficiency" is tricky. For example, an asynchronous HTTP server handler will actually respond more slowly than a synchronous handler. There's overhead to setting up the whole async thing with callbacks, etc. However, that slowdown AFAICT is unmeasurably small, and asynchronous code allows that server to handle more simultaneous requests than a synchronous server ever could (in real-world tests, we're talking 10x as a conservative estimate). Furthermore, asynchronous code is not limited by the thread injection rate of the thread pool, so asynchronous server code responds faster to sudden changes in load, reducing the number of request timeouts as compared to a synchronous server in the same scenario. Again, this is due to "async and threads", not "async instead of threads".

A few years ago, Node.js was heralded as an incredibly efficient server - based on real-world measurements. At the time, most ASP.NET apps were synchronous (writing asynchronous apps was quite hard before async, and companies knew it was cheaper to just pay for more server hardware). Node.js, in fact, only has one server thread that ever runs your app. It was 100% asynchronous, and that's where it got its scalability benefits from. ASP.NET took note of this, and ASP.NET Core (among other changes) made its entire stack asynchronous.