Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do I collect return values from Parallel.ForEach?

I'm calling a slow webservice in parallel. Things were great until I realized I need to get some information back from the service. But I don't see where to get the values back. I can't write to the database, HttpContext.Current appears to be null inside of a method called using Parallel.ForEach

Below is a sample program (in your mind, please imagine a slow web service instead of a string concatenation)

using System; using System.Threading.Tasks;  class Program {     static void Main(string[] args)     {         WordMaker m = new WordMaker();         m.MakeIt();     }     public class WordMaker     {         public void MakeIt()         {             string[] words = { "ack", "ook" };             ParallelLoopResult result = Parallel.ForEach(words, word => AddB(word));             Console.WriteLine("Where did my results go?");             Console.ReadKey();         }         public string AddB(string word)         {             return "b" + word;         }     }  } 
like image 266
MatthewMartin Avatar asked Sep 26 '12 21:09

MatthewMartin


People also ask

How do you break a parallel ForEach?

ForEach methods support cancellation through the use of cancellation tokens. For more information about cancellation in general, see Cancellation. In a parallel loop, you supply the CancellationToken to the method in the ParallelOptions parameter and then enclose the parallel call in a try-catch block.

Does ForEach work in parallel?

ForEach loop works like a Parallel. For loop. The loop partitions the source collection and schedules the work on multiple threads based on the system environment. The more processors on the system, the faster the parallel method runs.

Does parallel ForEach use ThreadPool?

Parallel. ForEach uses managed thread pool to schedule parallel actions. The number of threads is set by ThreadPool.


2 Answers

You've discarded it in here.

ParallelLoopResult result = Parallel.ForEach(words, word => AddB(word)); 

You probably want something like,

ParallelLoopResult result = Parallel.ForEach(words, word => {     string result = AddB(word);     // do something with result }); 

If you want some sort of collection at the end of this, consider using one of the collections under System.Collections.Concurrent, like ConcurrentBag

ConcurrentBag<string> resultCollection = new ConcurrentBag<string>(); ParallelLoopResult result = Parallel.ForEach(words, word => {     resultCollection.Add(AddB(word)); });  // Do something with the result 
like image 68
M Afifi Avatar answered Sep 22 '22 02:09

M Afifi


Your may consider using AsParallel extension method of IEnumerable, it will take care of the concurrency for you and collect the results.

words.AsParallel().Select(AddB).ToArray()

Synchronisation (e.g. locks or concurrent collections that use locks) are usually bottleneck of concurrent algorithms. The best is to avoid synchronisation as much as possible. I am guessing that AsParallel uses something smarter like putting all the items produced on single thread into a local non-concurrent collection and then combining these at the end.

like image 32
Steves Avatar answered Sep 23 '22 02:09

Steves