Does it make sense to spawn more than one thread per processor?

Question

From a logical point of view an application may need dozens or hundreds of threads, some of which will we sleeping most of the time, but a very few will be always running concurrently. The question is: Does it make any sense to spawn more concurrent threads than processors there are in a system, or it is a waste?

I've seen some server applications that implement a scheduler to logically manage tasks (often called jobs), but also spawning a lot of threads, so I don't see where the benefit is.

Thanks in advance.

Answer 1

Sure. If your software makes frequent use of disk or network IO, you can often improve throughput by adding a few more threads. Those extra threads will be awake and doing stuff while the other threads are blocking on IO.

Answer 2

Other have talked about situations in which it almost certainly does make sense (when you are doing any kind of slow IO).

It might not be a good idea if:

• your threads are doing CPU bound work

and

• the threads each want to use a lot (i.e. significant compared to the cache size) of memory that does not overlap

In this case there is the possibility of causing unnecessary cache misses.