Various ways of asynchronously returning a collection (with C# 7 features)

Question

I have a simple synchronous method, looking like this:

public IEnumerable<Foo> MyMethod(Source src)
{
    // returns a List of Oof objects from a web service
    var oofs = src.LoadOofsAsync().Result; 
    foreach(var oof in oofs)
    {
         // transforms an Oof object to a Foo object
         yield return Transform(oof); 
    }
}

Since the method is part of a web application, it is good to use all resources as effectively as possible. Therefore, I would like to change the method into an asynchronous one. The easiest option is to do something like this:

public async Task<IEnumerable<Foo>> MyMethodAsync(Source src)
{
    var oofs = await src.LoadOofsAsync();
    return oofs.Select(oof => Transform(oof));
}

I am not an expert on either async/await or IEnumerable. However, from what I understand, using this approach "kills" the benefits of IEnumerable, because the Task is awaited until the whole collection is loaded, thus omitting the "laziness" of the IEnumerable collection.

On other StackOverflow posts I have read several suggestions for using Rx.NET (or System.Reactive). Quickly browsing through the documentation I have read that IObservable<T> is their asynchronous alternative to IEnumerable<T>. However, using the naive approach and trying to type the following just did not work:

public async IObservable<Foo> MyMethodReactive(Source src)
{
    var oofs = await src.LoadOofsAsync();
    foreach(var oof in oofs)
    {
        yield return Transform(oof);
    }
}

I got an compilation error, that IObservable<T> does implement neither GetEnumerator(), nor GetAwaiter() - thus it cannot use both yield and async. I have not read the documentation of Rx.NET deeper, so I am probably just using the library incorrectly. But I did not want to spend time learning a new framework to modify a single method.

With the new possibilities in C# 7 it is now possible to implement custom types. Thus I, theoretically, could implement an IAsyncEnumerable, which would define both GetEnumerator() and GetAwaiter() methods. However, from my previous experience, I remember an unsuccessful attempt to create a custom implementation of GetEnumerator()... I ended up with a simple List, hidden in a container.

Thus we have 4 possible approaches to solve the task:

1. Keep the code synchronous, but with IEnumerable
2. Change it to asynchronous, but wrap IEnumerable in a Task<T>
3. Learn and use Rx.NET (System.Reactive)
4. Create a custom IAsyncEnumerable with C# 7 features

What are the benefits and drawbacks of each of these attempts? Which of them has the most significant impact on resource utilization?

Answer 1

• Keep the code synchronous, but with IEnumerable
• Change it to asynchronous, but wrap IEnumerable in a Task
• Learn and use Rx.NET (System.Reactive)
• Create a custom IAsyncEnumerable with C# 7 features

What are the benefits and drawbacks of each of these attempts? Which of them has the most significant impact on resource utilization?

In your situation, it sounds like the best option is Task<IEnumerable<T>>. Here's what where each option excels:

1. Synchronous code (or parallel synchronous code) excels when there is no I/O, but heavy CPU use. If you have I/O code waiting synchronously (like your first method implementation), the CPU is just burning cycles while waiting for the web service to respond doing nothing.

2. Task<IEnumerable<T>> is meant for when there is an I/O operation to fetch a collection. The thread running waiting for the I/O operation can then have something else scheduled on it while awaiting. This sounds like your case.

3. Rx is best for push scenarios: Where there is data being 'pushed' to your code which you want to respond to. Common examples are applications that receive stock-market pricing data, or chat applications.

4. IAsyncEnumerable is meant for when you have a collection where each item will require or generate an async task. An example: Iterating over a collection of items and executing some sort of unique DB query for each one. If your Transform was in fact an I/O-bound async method, then this is probably more sensible.