Aggregation of parallel for does not capture all iterations

Question

I have code that works great using a simple For loop, but I'm trying to speed it up. I'm trying to adapt the code to use multiple cores and landed on Parallel For.

At a high level, I'm collecting the results from CalcRoutine for several thousand accounts and storing the results in an array with 6 elements. I'm then re-running this process 1,000 times. The order of the elements within each 6 element array is important, but the order for the final 1,000 iterations of these 6 element arrays is not important. When I run the code using a For loop, I get a 6,000 element long list. However, when I try the Parallel For version, I'm getting something closer to 600. I've confirmed that the line "return localResults" gets called 1,000 times, but for some reason not all 6 element arrays get added to the list TotalResults. Any insight as to why this isn't working would be greatly appreciated.

object locker = new object();

Parallel.For(0, iScenarios, () => new double[6], (int k, ParallelLoopState state, double[] localResults) =>
            {
                List<double> CalcResults = new List<double>();
                for (int n = iStart; n < iEnd; n++)
                {
                    CalcResults.AddRange(CalcRoutine(n, k));
                }
                localResults = this.SumOfResults(CalcResults);
                return localResults;                   
            },
             (double[] localResults) =>
             {                    
                 lock (locker)
                {
                     TotalResults.AddRange(localResults);
                 }
             });

EDIT: Here's the "non parallel" version:

for (int k = 0; k < iScenarios; k++)            
{
          CalcResults.Clear();                
          for (int n = iStart; n < iEnd; n++)                
          {
              CalcResults.AddRange(CalcRoutine(n, k));                    
          }             
          TotalResults.AddRange(SumOfResults(CalcResults));
 }

The output for 1 scenario is a list of 6 doubles, 2 scenarios is a list of 12 doubles, ... n scenarios 6n doubles.

Also per one of the questions, I checked the number of times "TotalResults.AddRange..." gets called, and it's not the full 1,000 times. Why wouldn't this be called each time? With the lock, shouldn't each thread wait for this section to become available?

Answer 1

Check the documentation for Parallel.For

These initial states are passed to the first body invocations on each task. Then, every subsequent body invocation returns a possibly modified state value that is passed to the next body invocation. Finally, the last body invocation on each task returns a state value that is passed to the localFinally delegate

But your body delegate is ignoring the incoming value of localResults which the previous iteration within this task returned. Having the loop state being an array makes it tricky to write a correct version. This will work but looks messy:

//EDIT - Create an array of length 0 here    V    for input to first iteration
Parallel.For(0, iScenarios, () => new double[0],
    (int k, ParallelLoopState state, double[] localResults) =>
        {
            List<double> CalcResults = new List<double>();
            for (int n = iStart; n < iEnd; n++)
            {
                CalcResults.AddRange(CalcRoutine(n, k));
            }
            localResults = localResults.Concat(
                               this.SumOfResults(CalcResults)
                           ).ToArray();
            return localResults;                   
        },
         (double[] localResults) =>
         {                    
             lock (locker)
            {
                 TotalResults.AddRange(localResults);
             }
         });

(Assuming Linq's enumerable extensions are in scope, for Concat)

I'd suggest using a different data structure (e.g. a List<double> rather than double[]) for the state that more naturally allows more elements to be added to it - but that would mean changing SumOfResults that you've not shown. Or just keep it all a bit more abstract:

Parallel.For(0, iScenarios, Enumerable.Empty<double>(),
    (int k, ParallelLoopState state, IEnumerable<double> localResults) =>
    {
        List<double> CalcResults = new List<double>();
        for (int n = iStart; n < iEnd; n++)
        {
            CalcResults.AddRange(CalcRoutine(n, k));
        }
        return localResults.Concat(this.SumOfResults(CalcResults));                   
    },
     (IEnumerable<double> localResults) =>
     {                    
         lock (locker)
        {
             TotalResults.AddRange(localResults);
         }
     });

---

(If it had worked the way you seem to have assumed, why would they have you provide two separate delegates, if all it did, on the return from body, was to immediately invoke localFinally with the return value?)