Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Track progress when using TPL's Parallel.ForEach

What is the best way way to track progress in the following

long total = Products.LongCount();
long current = 0;
double Progress = 0.0;

Parallel.ForEach(Products, product =>
{
    try
    {
        var price = GetPrice(SystemAccount, product);
        SavePrice(product,price);
    }
    finally
    {
        Interlocked.Decrement(ref this.current);
    }});

I want to update the progress variable from 0.0 to 1.0 (current/total) but i don't want to use anything that would have an adverse effect on the parallelism.

like image 280
Madu Alikor Avatar asked Jan 26 '13 11:01

Madu Alikor


People also ask

Does parallel ForEach wait for completion?

You don't have to do anything special, Parallel. Foreach() will wait until all its branched tasks are complete. From the calling thread you can treat it as a single synchronous statement and for instance wrap it inside a try/catch.

Is parallel ForEach blocking?

No, it doesn't block and returns control immediately. The items to run in parallel are done on background threads.

Is parallel ForEach multiple threads?

ForEach is like the foreach loop in C#, except the foreach loop runs on a single thread and processing take place sequentially, while the Parallel. ForEach loop runs on multiple threads and the processing takes place in a parallel manner.


2 Answers

Jon's solution is good, if you need simple synchronization like this, your first attempt should almost always use lock. But if you measure that the locking slows things too much, you should think about using something like Interlocked.

In this case, I would use Interlocked.Increment to increment the current count, and change Progress into a property:

private long total;
private long current;
public double Progress
{
    get
    {
        if (total == 0)
            return 0;
        return (double)current / total;
    }
}

…

this.total = Products.LongCount();
this.current = 0;

Parallel.ForEach(Products, product =>
{
    try
    {
        var price = GetPrice(SystemAccount, product);
        SavePrice(product, price);
    }
    finally
    {
        Interlocked.Increment(ref this.current);
    }
});

Also, you might want to consider what to do with exceptions, I'm not sure that iterations that ended with an exception should be counted as done.

like image 171
svick Avatar answered Sep 30 '22 03:09

svick


Since you are just doing a few quick calculations, ensure atomicity by locking on an appropriate object:

long total = Products.LongCount();
long current = 0;
double Progress = 0.0;
var lockTarget = new object();

Parallel.ForEach(Products, product =>
{
    try
    {
        var price = GetPrice(SystemAccount, product);
        SavePrice(product,price);
    }
    finally
    {
        lock (lockTarget) {
            Progress = ++this.current / total;
        }
    }});
like image 40
Jon Avatar answered Sep 30 '22 04:09

Jon