C++ master/worker

Question

I am looking for a cross-platform C++ master/worker library or work queue library. The general idea is that my application would create some sort of Task or Work objects, pass them to the work master or work queue, which would in turn execute the work in separate threads or processes. To provide a bit of context, the application is a CD ripper, and the the tasks that I want to parallelize are things like "rip track", "encode WAV to Mp3", etc.

My basic requirements are:

• Must support a configurable number of concurrent tasks.
• Must support dependencies between tasks, such that tasks are not executed until all tasks that they depend on have completed.
• Must allow for cancellation of tasks (or at least not prevent me from coding cancellation into my own tasks).
• Must allow for reporting of status and progress information back to the main application thread.
• Must work on Windows, Mac OS X, and Linux
• Must be open source.

It would be especially nice if this library also:

• Integrated with Qt's signal/slot mechanism.
• Supported the use of threads or processes for executing tasks.

By way of analogy, I'm looking for something similar to Java's ExecutorService or some other similar thread pooling library, but in cross-platform C++. Does anyone know of such a beast?

Thanks!

Answer 1

I haven't used it in long enough that I'm not positive whether it exactly meets your needs, but check out the Adaptive Communications Environment (ACE). This library allows you to construct "active objects" which have work queues and execute their main body in their own threads, as well as thread pools that can be shared amoung objects. Then you can pass queue work objects on to active objects for them to process. Objects can be chained in various ways. The library is fairly heavy and has a lot to it to learn, but there have been a couple of books written about it and theres a fair amount of tutorial information available online as well. It should be able to do everything you want plus more, my only concern is whether it possesses the interfaces you are looking for 'out of the box' or if you'd need to build on top of it to get exactly what you are looking for.

Answer 2

I think this calls for intel's Threading Building Blocks, which pretty much does what you want.