Running Async Foreach Loop C# async await

Question

I am struggling to grasp the basic concept of c# async await.

Basically what I have is a List of objects which I need to process, the processing involves iterating through its properties and joining strings, and then creating a new object (in this case called a trellocard) and eventually adding a list of trellocards.

The iteration takes quiet a long time, So what I would like to do is process multiple objects at asynchronously.

I've tried multiple approaches but basically I want to do something like this. (in the below example I have removed the processing, and just put system.threading.thread.sleep(200). Im await that this is NOT an async method, and I could use tasks.delay but the point is my processing does not have any async methods, i want to just run the entire method with multiple instances.

    private async Task<List<TrelloCard>> ProcessJobs(IQueryable<IGrouping<CardGrouping, Job>> jobs)
    {
        List<TrelloCard> cards = new List<TrelloCard>();

        foreach (var job in jobs.ToList())
        {
           card = await ProcessCards(job, cards);  // I would like to run multiple instances of the processing
           cards.add(card); //Once each instance is finshed it adds it to the list
        }


  private async Task<TrelloCard> ProcessCards(Job job)
    {
        System.Threading.Thread.Sleep(2000);  //Just for examples sake

        return new TrelloCard();
    }

Answer 1

I am struggling to grasp the basic concept of c# async await.

Simple definition would be, Async-Await is a part .Net concurrency, which can be used to make multiple IO calls, and in process not waste the Threads, which are meant for Compute operations. Its like call to Database, Web service, Network calls, File IO, all of which doesn't need a current process thread

In your current case, where the use case is:

1. iterating through its properties and joining strings, and then creating a new object
2. eventually adding a list of trellocards

This seems to be a compute bound operation, until and unless you are doing an IO, to me it seems you are traversing an in memory object, for this case the better choice would be:

1. Parallel.ForEach, to parallelize the in memory processing, though you need to be careful of Race conditions, as a given memory could be accessed by multiple threads, thus corrupting it specially during write operation, so at least in current code use Thread safe collection like ConcurrentBag from System.Collections.Concurrent namespace, or which ever suit the use case instead of List<TrelloCard>, or you may consider following Thread safe list

Also please note that, in case your methods are not by default Async, then you may plan to wrap them in a Task.Run, to await upon, though this would need a Thread pool thread, but can be called using Async-Await

Parallel.Foreach code for your use case (I am doing direct replacement, there seems to be an issue in your code, since ProcessCards function, just takes Job object but you are also passing the collection Cards, which is compilation error):

 private List<TrelloCard> ProcessJobs(IQueryable<IGrouping<CardGrouping, Job>> jobs)
    {
        ConcurrentBag<TrelloCard> cards = new ConcurrentBag<TrelloCard>();

        Parallel.ForEach(jobs.ToList(), (job) => 
        {
           card = ProcessCards(job);  // I would like to run multiple instances of the processing
           cards.Add(card); //Once each instance is finshed it adds it to the list
        });

           return cards.ToList();
   }

  private TrelloCard ProcessCards(Job job)
    {
       return new TrelloCard();
    }

Answer 2

If you want them to run in parallel you could spawn a new Task for each operation and then await the completion of all using Task.WhenAll.

private async Task<List<TrelloCard>> ProcessJobs(IQueryable<IGrouping<CardGrouping, Job>> jobs)
{
    List<Task<TrelloCard>> tasks = new List<Task<TrelloCard>>();

    foreach (var job in jobs)
    {
        tasks.Add(ProcessCards(job));
    }

    var results = await Task.WhenAll(tasks);

    return results.ToList();
}


private Task<TrelloCard> ProcessCards(Job job)
{
    return Task.Run(() =>
    {
        System.Threading.Thread.Sleep(2000);  //Just for examples sake

        return new TrelloCard();
    });
}