How do you minimize the number of threads used in a tcp server application?

Question

I am looking for any strategies people use when implementing server applications that service client TCP (or UDP) requests: design patterns, implementation techniques, best practices, etc.

Let's assume for the purposes of this question that the requests are relatively long-lived (several minutes) and that the traffic is time sensitive, so no delays are acceptable in responding to messages. Also, we are both servicing requests from clients and making our own connections to other servers.

My platform is .NET, but since the underlying technology is the same regardless of platform, I'm interested to see answers for any language.

Answer 1

The modern approach is to make use of the operating system to multiplex many network sockets for you, freeing your application to only processing active connections with traffic.

Whenever you open a socket it's associated it with a selector. You use a single thread to poll that selector. Whenever data arrives, the selector will indicate the socket which is active, you hand off that operation to a child thread and continue polling.

This way you only need a thread for each concurrent operation. Sockets which are open but idle will not tie up a thread.

• Using the select() and poll() methods
• Building Highly Scalable Servers with Java NIO