TPL Parallel.For with long running tasks

Question

I am wanting to use the Task Parallel Library (TPL) in F# to execute many (>1000) long running tasks. Here is my current code:

Parallel.For(1, numberOfSets, fun j ->
    //Long running task here
    )

When I start this it appears that .NET initiates all of the tasks at once and bounces between them constantly. What would be better is if it stayed on a task until it is done before moving to the next one. This would minimize the context switching.

Is there a way to provide a hint to the scheduler? I know that it is possible to provide hints but I cannot find clear examples or is the scheduler already smart about this and it's just my perception that there are too many context switches occuring. Thanks for the help!

Answer 1

We had a similar problem - using C# instead than F#, but the libraries are the same. The solution was to limit the degree of parallelism:

ParallelOptions parallelOptions = new ParallelOptions();
parallelOptions.MaxDegreeOfParallelism = 16;
Parallel.For(0, n, parallelOptions, i => {
   . . . 
});

16 worked well for our tasks - you should experiment to see which value is better in your case.