Large number of simulteneous connections in thrift

Question

I'm trying to write a simple server with Thrift. At the beginning it looked promising, but I've stumbled into a problem with a number of clients connected at the same time. I'm using TThreadPoolServer, which allows 4 client to connect and then blocks other clients until I kill one from the connected. What can I do to allow more (possibly several hundreds) clients to be connected at the same time, without increasing the number of threads. I assumed that the worker threads allow to perform one client request at a time, but it looks like one thread handles one connection until it is closed. I'd like to avoid a situation when my clients has to reopen a socket to perform an action.

Answer 1

Taking another approach, if you are using C++ to build your server, you can use TNonblockingServer instead of TThreadPoolServer, which will allow you to accept many connections at once, regardless of how many threads are active, etc...

That being said, you won't necessarily be able to actually do work faster (handlers still execute in a thread pool), but more clients will be able to connect to you at once.

Here's what the code looks like for the NB server:

shared_ptr<TProtocolFactory> protocolFactory(new TBinaryProtocolFactory());
shared_ptr<MyHandler> handler(new MyHandler());
shared_ptr<TProcessor> processor(new MyProcessor(handler));
TNonblockingServer server(processor, protocolFactory, port);

Answer 2

Your limitation of four threads in the pool is built into the default constructor of the SimpleThreadManager:

class SimpleThreadManager : public ThreadManager::Impl {

 public:
  SimpleThreadManager(size_t workerCount=4, size_t pendingTaskCountMax=0) :
    workerCount_(workerCount),
    pendingTaskCountMax_(pendingTaskCountMax),
    firstTime_(true) {
  }
...
};

This ThreadManager object is passed to the ThreadPoolServer coonstructor, so pass a larger number to the constructor of this object to increase the size of your thread pool.