Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Parallel.ForEach and async-await

I had such method:

public async Task<MyResult> GetResult() {     MyResult result = new MyResult();      foreach(var method in Methods)     {         string json = await Process(method);          result.Prop1 = PopulateProp1(json);         result.Prop2 = PopulateProp2(json);      }      return result; } 

Then I decided to use Parallel.ForEach:

public async Task<MyResult> GetResult() {     MyResult result = new MyResult();      Parallel.ForEach(Methods, async method =>     {         string json = await Process(method);              result.Prop1 = PopulateProp1(json);         result.Prop2 = PopulateProp2(json);     });      return result; } 

But now I've got an error:

An asynchronous module or handler completed while an asynchronous operation was still pending.

like image 637
sreginogemoh Avatar asked Apr 17 '14 15:04

sreginogemoh


People also ask

Can we use async in parallel ForEach?

Foreach itself is very useful and efficient for most operations. Sometimes special situations arise where high latency in getting data to iterate over, or processing data inside the foreach depends on an operation with very high latency or long processing.

Does parallel ForEach wait?

You don't have to do anything special, Parallel. Foreach() will wait until all its branched tasks are complete. From the calling thread you can treat it as a single synchronous statement and for instance wrap it inside a try/catch.

When should you use parallel ForEach?

The Parallel. ForEach method splits the work to be done into multiple tasks, one for each item in the collection. Parallel. ForEach is like the foreach loop in C#, except the foreach loop runs on a single thread and processing take place sequentially, while the Parallel.

Does ForEach work in parallel?

ForEach loop works like a Parallel. For loop. The loop partitions the source collection and schedules the work on multiple threads based on the system environment. The more processors on the system, the faster the parallel method runs.


1 Answers

async doesn't work well with ForEach. In particular, your async lambda is being converted to an async void method. There are a number of reasons to avoid async void (as I describe in an MSDN article); one of them is that you can't easily detect when the async lambda has completed. ASP.NET will see your code return without completing the async void method and (appropriately) throw an exception.

What you probably want to do is process the data concurrently, just not in parallel. Parallel code should almost never be used on ASP.NET. Here's what the code would look like with asynchronous concurrent processing:

public async Task<MyResult> GetResult() {   MyResult result = new MyResult();    var tasks = Methods.Select(method => ProcessAsync(method)).ToArray();   string[] json = await Task.WhenAll(tasks);    result.Prop1 = PopulateProp1(json[0]);   ...    return result; } 
like image 51
Stephen Cleary Avatar answered Oct 08 '22 19:10

Stephen Cleary