TBB vs. Homegrown Workqueue

Question

I know TBB (Thread Building Blocks) claim to have a sophisticated engine, but from the algorithmic point of view:

If we had (say on Linux) a workqueue that has N working-threads (POSIX threads, N is the number of cores) and a mutex-synchronized queue of tasks, each working thread then taking a task from the queue when idle, also some synchronization calls, what else could TBB offer, not counting nice C++ syntax? I don't see a better algorithm than greedy assignment of tasks to cores.

Answer 1

As somebody who has developed their own work-stealing scheduler, I can say the following:

• Don’t write your own scheduler (and a work-queue counts here).
• You’ll either do it inefficiently, or you’ll do it wrong.

In fact, it’s not that hard to write a correct scheduler. Unfortunately, it is hard if you want to do it efficiently. An efficient scheduler effectively precludes the use of locks (except perhaps in very specific, well-specified situations) and lock-free cross-thread communication is a world of pain.

As an anecdote, I actually implemented one scheduler where I essentially had to copy the existing algorithm into code and I still managed to introduce almost any race condition imaginable into the code. Debugging this code was a mixture of

• writing huge, convoluted test cases (just to pick up the occasional failure which only occurred in < 1% of the runs),
• spending hours on end just staring at the code, trying to figure out the error by applying logic
• tracing each single line in the debugger (which would crash without stack trace once an error occurred), keeping track of the state of all variables in all threads manually just to be sure that the actual state of the program matched the expected state
• reducing the code several times essentially down to zero and rebuilding, commenting out single lines or pairs of lines to see the effect (huge combinatorial space), and
• running against walls, head first.