Best way to start several async tasks in parallel? [closed]

Question

I have potentially several thousand independent tasks that need to be run. Each of them may make database calls, so they're already leveraging async where possible. That said, if I wanted all of them to run in parallel what's the best way to do this?

I've got this currently:

Parallel.For(0, items.Count, async _ => await PerformTask());

I've also considered using:

List<Task> tasks = new List<Task>();
for(var i = 0; i < items.Count; ++i) tasks.Add(PerformTask());
await Task.WhenAll(tasks); // or possibly Task.WaitAll(tasks.ToArray())

Is there an objectively best way to do this?

Edit: This is different from the marked duplicate question since I'm not asking the difference. I'm asking which way is correct for my use case.

Answer 1

Parallel is not an option because you have asynchronous actions.

Your options are:

• Start all of the tasks simultaneously, and then use await Task.WhenAll for them all to complete. You can use SemaphoreSlim if you find you need to throttle the number of active tasks.
• Use an ActionBlock<T> (from TPL Dataflow) to queue up the work individually. You can use ExecutionDataflowBlockOptions.MaxDegreeOfParallelism if you want to process more than one simultaneously.

The ActionBlock<T> approach would be better if you don't know about all the tasks at the time they're started (i.e., if more can arrive while you're processing), or if other nearby parts of your code will fit into a "pipeline" kind of design.

Task.WhenAll is nice because it doesn't require a separate library with its own design philosophy and learning curve.

Either Task.WhenAll or ActionBlock<T> would work well for your use case.