TCP/IP - Solving the C10K with the thread per client approach

Question

After reading the famous C10k article and searching on the web about how things have evolved since it was written, I would like to know if it would be possible for a today's standard server to handle >10000 concurrent connections using a thread per connection (possibly with the help of a pool of threads to avoid the creation/killing process).

---

Some details that may affect the approach to the problem:

1. Input, intermediate processing and output.
2. Length of each connection.
3. Technical specifications of the server (cores, processors, RAM, etc...)
4. Combining this system with alternative techniques like AIO, polling, green threads, etc...

---

Obviously I'm not an expert in the matter, so any remarks or advices will be highly appreciated :)

Answer 1

Absolutely. A standard server can handle more than 10K concurrent connections using the model with one thread per connection. I have build such an application, and five years ago, it was running with more than 50K concurrent connections per process on a standard Linux server. Nowadays, it should be possible to run the same application with more than 250K concurrent connections on current hardware.

There are only a few things to keep in mind:

• Reuse threads by using a thread pool. There is no need to kill threads if they are not used, because the resource usage should be optimized for peak loads.
• Stack size: By default each Linux thread reserves 8 MB for its stack. That sums up to 80 GB for 10K threads. You should set the default stack size to some value between 64k and 512k, which isn't a problem, because most applications don't require deeper call stacks.
• If the connections are short-lived, optimize for new connections by creating several sockets on the same endpoint with the option SO_REUSEPORT.
• Increase the user limits: open files (default 1.024), max user processes
• Increase system limits, e.g. /proc/sys/kernel/pid_max (default 32K), /proc/sys/kernel/threads-max, and /proc/sys/vm/max_map_count (default 65K).

The application mentioned above was initially designed to handle only 2K concurrent connections. However, with the growth in use, we didn't have to make significant changes to the code in order to scale up to 50K connections.

Answer 2

You might like a recent follow-up on the subject: The Secret To 10 Million Concurrent Connections -The Kernel Is The Problem, Not The Solution.