One thread per client. Doable?

Question

I'm writing a Java server which uses plain sockets to accept connections from clients. I'm using the fairly simple model where each connection has its own thread reading from it in blocking mode. Pseudo code:

handshake();

while(!closed) {
  length = readHeader(); // this usually blocks a few seconds
  readMessage(length);
}

cleanup();

(Threads are created from an Executors.newCachedThreadPool() so there shouldn't be any significant overhead in starting them)

I know this is a bit of a naive setup and it wouldn't really scale well to many connections if the threads were dedicated OS threads. However, I've heard that multiple threads in Java can share one hardware thread. Is that true?

Knowing that I'll be using the Hotspot VM on Linux, on a server with 8 cores and 12GB of RAM, do you think this setup will work well for thousands of connections? If not, what are the alternatives?

Answer 1

This will scale well for up to hundreds of connections, not to thousands. One issue is that a Java thread takes quite a bit of stack as well (e.g. 256K), and the OS will have problems scheduling all your threads.

Look at Java NIO or framworks that will help you get started doing complex stuff more easily (e.g. Apache Mina)

Answer 2

It is possible this will scale to thousands of clients. But how many thousands is the next question.

A common alternative is to use Selectors and non-blocking I/O found in the java.nio package.

Eventually you get into the question of whether it's useful to set up your server in a clustered configuration, balancing the load over multiple physical machines.