How to choose correct number of threads for C++ multithread application?

Question

I am C++ backend developer. I develop server side for realtime game. So, application architecture look like this:

1) I have class Client, which process requests from game client. Examples of requests: login, buy something in store (game internal store), or make some stuff. Also this Client handle user input events from game client (It's very often events, which sends ten times in second from game client to server, when player play in gameplay).

2) I have thread pool. When game client connect to server I create Client instance and bind them to one of threads from pool. So, we have relationships one to many: one thread - many Clients. Round-robin used to chose thread for binding.

3) I use Libev to manage all events inside server. It's mean when Client instance receive some data from game client through network, or handle some request, or trying to send some data through network to game client he lock hi's thread. While he make some stuff other Clients, which share same thread will be locked.

So, thread pool is bottleneck for application. To increase number concurrent players on server, who will play without lags I need increase number of threads in thread pool.

Now application work on server with 24 logical cpus ( cat /proc/cpuinfo say it). And I set thread pool size to 24 (1 processor - 1 thread). It's mean, that with current online 2000 players every thread serves about 84 Client instances. top say that processors used less then 10 percents.

Now question. If I increase number of threads in thread pool is It increase or decrease server performance (Context switching overhead vs locked Clients per thread)?

UPD 1) Server has async IO (libev + epoll), so when I says that Client locked when send and receive data I mean coping to buffers. 2) Server also has background threads for slow tasks: database operations, hard calculation operations, ...

Answer 1

Well few issues.

2) I have thread pool. When game client connect to server I create Client instance and bind them to one of threads from pool. So, we have relationships one to many: one thread - many Clients. Round-robin used to chose thread for binding.

You didn't mention asynchronous IO in any of the points, I believe your true bottleneck here is not the thread count, but the fact that a thread is blocked because of an IO action. by using asynchronous IO (which is not an IO action on another thread) - the thoughput of your server increases by huge magnitutes.

3) I use Libev to manage all events inside server. It's mean when Client instance receive some data from game client through network, or handle some request, or trying to send some data through network to game client he lock hi's thread. While he make some stuff other Clients, which share same thread will be locked.

again, without asynchronous IO this architecture is very much 90's server-side architecture (a-la Apache style). for maximum performance, your threads should only do CPU bound tasks and should not wait for any IO actions.

So, thread pool is bottleneck for application. To increase number concurrent players on server, who will play without lags I need increase number of threads in thread pool.

Dead wrong. read about the 10k concurrency issue.

Now question. If I increase number of threads in thread pool is It increase or decrease server performance (Context switching overhead vs locked Clients per thread)?

So, the anecdote about number of threads as the number of cores is only valid when your threads do only cpu bound tasks and they are never blocked and they ae 100% staurated with cpu tasks. if your threads are also being blocked by locks or IO actions, this fact breaks.

If we take a look at common Server side architectures, we can determine what the best design we need

Apache style architecture:
having a fixed-size thread pool. assigining a thread to each connection in the connection queue. non asynchronous IO.
pros: non.
cons: extremly bad throughput

NGNix/Node.js architecture:
having mono-threaded - multi-processed application. using asynchronous IO.
pros: simple architecture that eliminates multi-threaded problems. goes extremly well with servers that serve static data.
cons: if the processes have to shrare data, huge amount of CPU time is burned on serilizeing-passing-desirilizing data between processes. also, multithreaded application can increases performance if done correctly.

Modern .Net architecure:
having multi threaded-mono processed application. using asynchronous IO.
pros: if done correctly, the performance can blast!
cons: it's somewhat tricky to tune multi-threaded applicaiton and use it without corrupting shrared data.

So to sum it up, I think that in your specific case you should defenitly use asynchronous IO only + having a threadpool with the number of threads equal to the number of cores.

If you're using Linux, Facebook's Proxygen can manage everything we talked about (multithreaded code with asynchronous IO) beautifuly for you. hey, facebook are using it!