Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to do proper Parallel.ForEach, locking and progress reporting

I'm trying to implement the Parallel.ForEach pattern and track progress, but I'm missing something regarding locking. The following example counts to 1000 when the threadCount = 1, but not when the threadCount > 1. What is the correct way to do this?

class Program
{
   static void Main()
   {
      var progress = new Progress();
      var ids = Enumerable.Range(1, 10000);
      var threadCount = 2;

      Parallel.ForEach(ids, new ParallelOptions { MaxDegreeOfParallelism = threadCount }, id => { progress.CurrentCount++; });

      Console.WriteLine("Threads: {0}, Count: {1}", threadCount, progress.CurrentCount);
      Console.ReadKey();
   }
}

internal class Progress
{
   private Object _lock = new Object();
   private int _currentCount;
   public int CurrentCount
   {
      get
      {
         lock (_lock)
         {
            return _currentCount;
         }
      }
      set
      {
         lock (_lock)
         {
            _currentCount = value;
         }
      }
   }
}
like image 496
John Landheer Avatar asked Jan 24 '13 11:01

John Landheer


People also ask

How do you continue in parallel ForEach?

ForEach functionally equivalent to continue in a foreach loop? Parallel. ForEach(items, parallelOptions, item => { if (! isTrue) continue; });

How do I limit the number of threads in parallel ForEach?

We can restrict the number of concurrent threads created during the execution of a parallel loop by using the MaxDegreeOfParallelism property of ParallelOptions class.

Does parallel ForEach use ThreadPool?

Parallel. ForEach uses managed thread pool to schedule parallel actions. The number of threads is set by ThreadPool.

Is parallel ForEach faster than ForEach?

The execution of Parallel. Foreach is faster than normal ForEach.


2 Answers

The usual problem with calling something like count++ from multiple threads (which share the count variable) is that this sequence of events can happen:

  1. Thread A reads the value of count.
  2. Thread B reads the value of count.
  3. Thread A increments its local copy.
  4. Thread B increments its local copy.
  5. Thread A writes the incremented value back to count.
  6. Thread B writes the incremented value back to count.

This way, the value written by thread A is overwritten by thread B, so the value is actually incremented only once.

Your code adds locks around operations 1, 2 (get) and 5, 6 (set), but that does nothing to prevent the problematic sequence of events.

What you need to do is to lock the whole operation, so that while thread A is incrementing the value, thread B can't access it at all:

lock (progressLock)
{
    progress.CurrentCount++;
}

If you know that you will only need incrementing, you could create a method on Progress that encapsulates this.

like image 66
svick Avatar answered Sep 20 '22 13:09

svick


Old question, but I think there is a better answer.

You can report progress using Interlocked.Increment(ref progress) that way you do not have to worry about locking the write operation to progress.

like image 43
Roy T. Avatar answered Sep 18 '22 13:09

Roy T.