MaxDegreeOfParallelism deciding on optimal value

Question

Simple question.

How do you decide what the optimal value for MaxDegreeOfParallelism is for any given algorithm? What are the factors to consider and what are the trade-offs?

Answer 1

I think it depends, if all your tasks are "CPU bound" it would be probably equal to the number of CPUs in your machine. Nevertheless if you tasks are "IO bound" you can to start to increase the number.

When the CPU has to switch from one thread to other (context switch) it has a cost, so if you use too much threads and the CPU is switching all the time, you decrease the performance. In the other hand, if you limit that parameter too much, and the operations are long "IO bound" operations, and the CPU is idle a lot of time waiting for those tasks to complete... you are not doing the most with your machine's resources (and that is what multithreading is about)

I think it depends on each algorithm as @Amdahls's Law has pointed out, and there is not a master rule of thumb you can follow, you will have to plan it and tun it :D

Cheers.

Answer 2

For local compute intensive processes you should try two values;

• number of physical cores or processors
• number of virtual cores (physical including hyperthreading)

One of these is optimal in my experience. Sometimes using hyperthreading slows down, usually it helps. In C# use Environment.ProcessorCount to find the number of cores including hyperthreading fake cores. Actual physical cores is more difficult to determine. Check other questions for this.

For processes that have to wait for resources (db queries, file retrieval) scaling up can help. If you have to wait 80% of the time a process is busy the rule of thumb is to start 5 threads for it so one thread will be busy at any time. Maximizing the 5x20% each process requires locally.