How to optimize number of threads per number of cores

Question

I'm trying to get a better idea of how many threads should run on n number of cores. I know this a complicated question in which the answer depends on a number of factors, such as how much shared state there is, and how much sleeping and waiting on resources each thread does.

To simplify things, let's say we have 2 cores and just one process that can divide its work into threads with no shared state. Let's say each thread just performs computation after computation with no sleeping and no waiting on resources. Would the ideal number of threads in this case be 2?

Let's complicate things a bit and say that the threads have to do some sort of disk I/O. How does this change our answer? I would think that we could have more than 2 cores in this case.

Or let's say they don't do any sleeping or waiting on resources, but instead there's some memory that they both have access to that requires synchronization. How does this change our answer? I would think that in this case we may actually prefer 1 thread over 2, depending on how much synchronization is required.

Answer 1

This is a hard question to answer in the general matter. It really depends on more specifics in the case. What to remember is that it costs to do context-switches - if you're only doing computations it would be wasteful to have 2 threads running on one core (as you wouldn't really gain anything - only lose in the context switches). On the other hand if you are waiting for resources and at the same time can continue with other calculations, it is a good idea to have a thread wait for those resources to not make the entire execution lag behind.