CPU usage vs Number of threads

Question

In general what is the relation between CPU usage and number of threads in a program. Assumptions:

• Multi-core CPU
• Threads do the exact same job (assume they fetch identical work items from a queue and process them)

Answer 1

It depends on the nature of the application.

• An application that mostly do calculations - a ratio of 1 thread per core is a reasonable decision, since you don't want to spawn too many threads due to overhead, and you want to take advantage of all your cores.
• An application that mostly do IO operations (like http requests) can spawn much more threads then the #cores and still increase efficiency, since the bottleneck is the waiting time per IO request, and you want to gain as much information as possible in each time you need to wait.

That said, the CPU-usage you are going to get is still dependent on many factors (IO, synchronization, non parallel parts in your program).

If you are interested in the speed the application will take - always remember Amdahl's law, which gives you a strict bound on the time (speed-up) your application is going to take, even when having infinite number of working cores.

Answer 2

There is no such general relationship, except for the obvious ones:

• an application can't use more CPU time (CPU seconds) than the number of available cores multiplied by the number of (wall clock) seconds that it runs, and
• a single thread can't use more than one CPU second per second.

The actual amount of CPU that a multi-threaded application depends mostly on the nature of the application, and the way that you've implemented it:

• If the computation performed by each thread does not generate contention with other threads for locks, memory access and so on, then you should be able to approach the theoretical limit of available CPU resources.

• Contention is liable to reduce effective CPU usage, sometimes dramatically.

But there are no general formulae that will tell you how much speed-up you can get.

Answer 3

I think there is no relation or not easy one. It depends on the jobs the threads are doing. A program with one thread can consume 100% of CPU and a program with lots of threads can consume less.

If you are looking for an optimized relation between threads and job done, you must study your case, and possibly found an empiric solution.