Can i use boost::threadpool as a 'thread-safe queue'?

Question

What I need is actually a thread-safe queue structure, where multiple clients keep dumping data into the queue and one working thread keeps processing and popping the queue

is there any well-established solution existing in STL or Boost?

I now think about using Boost::threadpool to do this. Simply set the number of parallel threads to be 1, the input parameter of task function is changed every time new message arrives from a client. Does this make sense, is there any limitation that I have not yet anticipated here?

Answer 1

In boost there is a message queue class, that is what you need: a thread-safe queue.

Message queues is a widely-used concept for interprocess communication. A message queue is thread-safe queue, which key feature is that it blocks on reading from empty queue and waits for data to appear in it. In that boost class, timed waits are also supported, as well as blocking the writer if the queue is full.

Answer 2

If you need such a framework in a single process application, boost::asio::io_service should be sufficient. Here is a working box sample class using boost::thread and boost::asio::io_service.

#include <boost/asio.hpp>
#include <boost/bind.hpp>
#include <boost/thread.hpp>

class IWorkerThreadJob
{
    public:
        virtual ~IWorkerThreadJob(){};
        virtual void execute() = 0;
};


class BoostBasedWorkingBox
{
    public:

        BoostBasedWorkingBox():
            m_IOServiceWork(m_IOService), // Give some work to io_service or else it will simply return from ::run method .
            m_WorkerThread(boost::bind(&boost::asio::io_service::run, &m_IOService))
        {}

        ~BoostBasedWorkingBox()
        {
            m_IOService.stop();
            m_WorkerThread.join();
        }

        void processJob(IWorkerThreadJob* pJob)
        {
            m_IOService.post(boost::bind(&IWorkerThreadJob::execute,pJob));
        }   

    protected:
        boost::thread m_WorkerThread;
        boost::asio::io_service m_IOService;
        boost::asio::io_service::work m_IOServiceWork;


}

Use:- Implement IWorkerThreadJob interface. Call processJob method from multiple clients.

Here boost::asio::io_service acts as a thread safe queue.

Answer 3

If you are on Windows you can use concurrent_queue in ppl.h (new for VS2010). If you aren't on windows you can use concurrent_queue.h in Intel's thread building blocks.

Anthony Williams also has a queue based on the condition variable on his blog which is good.