Nested Parallel.For() loops speed and performance

Question

I have a nested for loop. I have replaced the first For with a Parallel.For() and the speed of calculation increased.

My question is about replacing the second for (inside one) with a Parallel.For(). Will it increase the speed? or there is no difference? or it will be slower?

Edit:

Since the cores are not unlimited (usually there is 2 to 8 cores), the inside loop is running parallel. So, if I change the inside for with a Parallel.For(), again it runs parallel. But i'm not sure how it changes the performance and speed.

Answer 1

From "Too fine-grained, too coarse-grained" subsection, "Anti-patterns" section in "Patterns of parallel programming" book by .NET parallel computing team:

The answer is that the best balance is found through performance testing. If the overheads of parallelization are minimal as compared to the work being done, parallelize as much as possible: in this case, that would mean parallelizing both loops. If the overheads of parallelizing the inner loop would degrade performance on most systems, think twice before doing so, as it’ll likely be best only to parallelize the outer loop.

Take a look at that subsection, it is self-contained with detailed examples from parallel ray tracing application. And its suggestion of flattening the loops to have better degree of parallelism may be helpful for you too.

Answer 2

It again depends on many scenarios,

1. Number of parallel threads your cpu can run.
2. Number of iterations.

If your CPU is a single-core processor, you will not get any benefits.

If the number of iterations is greater, you will get some improvements.

If there are just a few iterations, it will be slow as it involves extra overload.