Update - changed the title of the question to reflect what I'm really after
Consider the following piece of code:
// this query generates 12 instances of Func<int>, which each when executed
// print something to the console and wait for 1 second.
var actions = Enumerable.Range(0, 12).Select(i => new Func<int>(() =>
{
Console.WriteLine("{0} - waiting 1 sec", i);
Thread.Sleep(1000);
return 1;
}));
// define a parallel query. Note the WithDegreeOfParallelism call here.
var query = from action in actions.AsParallel().WithDegreeOfParallelism(12)
select action();
// execute, measuring total duration
var stopw = Stopwatch.StartNew();
query.ToList();
Console.WriteLine(stopw.Elapsed);
Console.WriteLine(Environment.ProcessorCount); // 3 on my machine
When omitting the call to WithDegreeOfParallelism
, this executes in 4 chunks, taking about 4 seconds in total, which is what I would expect since my CPU count is 3.
However, when calling WithDegreeOfParallelism
with any number above 4, I always get 3 chunks, and the total duration does not go under 3 seconds. I would expect that a value of 12 would get a total duration of (a little more than) 1 second.
What am I missing? And how can I enforce the parallel execution of more than 4 non-CPU intensive tasks, which is what I'm after?
Update: I could of course go back to manually spinning up threads, but I was hoping that the new PFX library would make this a bit easier... Anyway, the code below gives me about 1 second total execution time
List<Thread> threads = new List<Thread>();
for (int i = 0; i < 12; i++)
{
int i1 = i;
threads.Add(new Thread(() =>
{
Console.WriteLine(i1);
Thread.Sleep(1000);
}));
}
threads.ForEach(t => t.Start());
threads.ForEach(t => t.Join());
Try starting new tasks in your parallel loop with the option TaskCreationOptions.LongRunning
. They will start right away, instead of waiting until a thread on the threadpool becomes available.
As I said WithDegreeOfParallelism is setting only a upper bound.Try increasing your tasks from 10 to 100. You will ended up with around 10 seonds for all 100 of them. Your code is good for larger number of tasks having smaller operations.
and add Console.WriteLine("{0} threads " ,Process.GetCurrentProcess().Threads.Count);
inside your task then you can see how many threads are created.( Thread count is not the count of plinq created threads. See how it increasing).
There are lots of ways to do the parallelism with PLinq . Read this article http://msdn.microsoft.com/en-us/library/dd997411.aspx. You need to choose best way for the relevant requirement to get better performance.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With