List of objects with async Task methods, execute all concurrently

Question

Given the following:

BlockingCollection<MyObject> collection;

public class MyObject
{
   public async Task<ReturnObject> DoWork() 
   {       
      (...) 
      return await SomeIOWorkAsync();
   }
}

What would be the correct/most performant way to execute all DoWork() tasks asynchronously on all MyObjects in collection concurrently (while capturing the return object), ideally with a sensible thread limit though (I believe the Task Factory/ThreadPool does some management here)?

Answer 1

You can make use of the WhenAll extension method.

var combinedTask = await Task.WhenAll(collection.Select(x => x.DoWork());

It will start all tasks concurrently and waits for all to finish.

Answer 2

ThreadPool manages the number of threads running, but that won't help you much with asynchronous Tasks.

Because of that, you need something else. One way to do this is to utilize ActionBlock from TPL Dataflow:

int limit = …;
IEnumerable<MyObject> collection = …;

var block = new ActionBlock<MyObject>(
    o => o.DoWork(),
    new ExecutionDataflowBlockOptions { MaxDegreeOfParallelism = limit });

foreach (var obj in collection)
    block.Post(o);

block.Complete();
await block.Completion;