Run multiple instances of same method asynchronously?

Question

My requirement is quite weird.

I have SomeMethod() which calls GetDataFor().

public void SomeMethod()
{
    for(int i = 0; i<100; i++) {
        var data = GetDataFor(i);
    }
}

public data GetDataFor(int i) {
    //call a remote API
    //to generate data for i
    //store to database
    return data;
}

For each i, the end result will always be different. There is no need to wait for GetDataFor(i) to complete before calling GetDataFor(i+1).

In other words I need to:

• call GetDataFor() for each i+1 immediately after successfully calling i (Calling them in parallel looks impossible)
• wait until all the 100 instances of GetDataFor() are completed running
• leave the scope of SomeMethod()

Following YK1's answer, I have tried to modify it like this:

public async Task<void> SomeMethod()
{
    for(int i = 0; i < 100; i++) {
        var task = Task.Run(() => GetDataFor(i));
        var data = await task;
    }
}

It didn't thrown any errors but I need to understand the concept behind this:

• How task will distinguish between different calls for awaiting? It is getting over-written.
• Is it blatantly wrong way to do this? So, how do do it right?

Answer 1

You can use Parallel.For:

public void SomeMethod()
{
    Parallel.For(0, 100, i =>
    {
        var data = GetDataFor(i);
        //Do something
    });
}

public data GetDataFor(int i)
{
    //generate data for i
    return data;
}

EDIT:

The syntax of a parallel loop is very similar to the for and foreach loops you already know, but the parallel loop runs faster on a computer that has available cores. Another difference is that, unlike a sequential loop, the order of execution isn't defined for a parallel loop. Steps often take place at the same time, in parallel. Sometimes, two steps take place in the opposite order than they would if the loop were sequential. The only guarantee is that all of the loop's iterations will have run by the time the loop finishes.

For parallel loops, the degree of parallelism doesn't need to be specified by your code. Instead, the run-time environment executes the steps of the loop at the same time on as many cores as it can. The loop works correctly no matter how many cores are available. If there is only one core, the performance is close to (perhaps within a few percentage points of) the sequential equivalent. If there are multiple cores, performance improves; in many cases, performance improves proportionately with the number of cores.

You can see a more detailed explanation here.

Answer 2

There's a couple of different approaches.

First, you could keep it synchronous and just execute them in parallel (on different threads). Parallel LINQ is better than Parallel if you want to collect all the results in the calling method before continuing:

public data[] SomeMethod()
{
  return Enumerable.Range(0, 100)
      .AsParallel().AsOrdered()
      .Select(GetDataFor).ToArray();
}

Second, you could make it asynchronous. To make something truly asynchronous, you need to start at the lowest level (in this case, "call a remote API" and "store to database") and make that asynchronous first. Then you can make GetDataFor asynchronous:

public async Task<data> GetDataForAsync(int i)
{
  await .. //call a remote API asynchronously
  await .. //store to database asynchronously
  return data;
}

Then you can make SomeMethod asynchronous as well:

public Task<data[]> SomeMethodAsync()
{
  return Task.WhenAll(
      Enumerable.Range(0, 100).Select(GetDataForAsync)
  );
}

Making the code asynchronous is more work - more of the code has to change - but it's better in terms of scalability and resource use.