Say I have an IO-bound task. I'm using WithDegreeOfParallelism = 10 and WithExecution = ForceParallelism mode, but still the query only uses two threads. Why?
I understand PLINQ will usually choose a degree of parallelism equal to my core count, but why does it ignore my specific request for higher parallelism?
static void Main(string[] args)
{
TestParallel(0.UpTo(8));
}
private static void TestParallel(IEnumerable<int> input)
{
var timer = new Stopwatch();
timer.Start();
var size = input.Count();
if (input.AsParallel().
WithDegreeOfParallelism(10).
WithExecutionMode(ParallelExecutionMode.ForceParallelism).
Where(IsOdd).Count() != size / 2)
throw new Exception("Failed to count the odds");
timer.Stop();
Console.WriteLine("Tested " + size + " numbers in " + timer.Elapsed.TotalSeconds + " seconds");
}
private static bool IsOdd(int n)
{
Thread.Sleep(1000);
return n%2 == 1;
}
PLINQ tries to find the optimal number of threads to perform what you want it to do as quickly as possible, if you only have 2 cores on your cpu, that number is most likely 2. If you had a quad core, you would be more likely to see 4 threads appear, but creating 4 threads on a dual core machine wouldn't really improve performance because only 2 threads could be active at the same time.
Also, with IO-based operations, it is likely that any extra threads would simply block on the first IO operation performed.
10 is maximum
Sets the degree of parallelism to use in a query. Degree of parallelism is the maximum number of concurrently executing tasks that will be used to process the query.
From here:
MSDN
It appears PLINQ tunes the number of threads. When I wrapped the above code in a while(true) loop, the first two iteration took two seconds to run, but the third and above took only one second. PLINQ understood the cores are idle and upped the number of threads. Impressive!
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With