Is it possible to "await yield return DoSomethingAsync()"

Question

What you are describing can be accomplished with the Task.WhenAll method. Notice how the code turns into a simple one-liner. What happens is that each individual url begins downloading and then WhenAll is used combine those operations into a single Task which can be awaited.

Task<IEnumerable<string>> DownLoadAllUrls(string[] urls)
{
    return Task.WhenAll(from url in urls select DownloadHtmlAsync(url));
}

tl;dr Iterators as implemented with yield are a blocking construct, so as of right now await and yield are incompatible.

Long Because iterating over an IEnumerable is a blocking operation, calling a method marked as async will still execute it in a blocking manner, since it has to wait for that operation to finish.

async Task<IEnumerable<Foo>> Method(String [] Strs)
{
  foreach(var str in strs)
  {
    yield return await DoSomethingAsync( str)
  }
}  

The awaiting Method mixes meanings. Do you want to wait until the Task has an IEnumerable and then block on iterating over it? Or are you trying to await each value of the IEnumerable?

I assume the second is the desired behavior and in that case the existing Iterator semantics will not work. The IEnumerator<T> interface is basically

public interface IEnumerator<T>
  T Current;
  bool MoveNext();
}

I'm ignoring Reset() since it makes no sense for a sequence of asynchronous results. But what you would need is something like this:

public interface IAsyncEnumerator<T>
  T Current;
  Task<bool> MoveNext();
}

Of course, foreach also won't work with this and you'd have to iterate manually like this:

var moveNext = await asyncEnumerator.MoveNext();
while(moveNext) {

  // get the value that was fetche asynchronously
  var v = asyncEnumerator.Current;

  // do something with that value

  // suspend current execution context until next value arrives or we are done
  moveNext = await asyncEnumerator.MoveNext();
}

According to the new features at C# 8.0 (link#1 and link#2) we'll have IAsyncEnumerable<T> interface support that will allow to implement your second attempt. It will look like this:

async Task<Foo> DoSomethingAsync(string url)
{
    ...
}       
// producing IAsyncEnumerable<T>
async IAsyncEnumerable<Foo> DownLoadAllURL(string[] strs)
{
    foreach (string url in strs)
    {
        yield return await DoSomethingAsync(url);
    }
}
...
// using
await foreach (Foo foo in DownLoadAllURL(new string[] { "url1", "url2" }))
{
    Use(foo);
}

We can achieve the same behavior at C# 5 but with a different semantics:

async Task<Foo> DoSomethingAsync(string url)
{
    ...
}
IEnumerable<Task<Foo>> DownLoadAllURL(string[] strs)
{
    foreach (string url in strs)
    {
        yield return DoSomethingAsync(url);
    }
}

// using
foreach (Task<Foo> task in DownLoadAllURL(new string[] { "url1", "url2" }))
{
    Foo foo = await task;
    Use(foo);
}

Brian Gideon's answer implies that the calling code will get asynchronously a collection of results that were obtained in parallel. The code above implies that the calling code will get results like from a stream one by one in asynchronous manner.

I know that I'm too late with the answer, but here is another simple solution that can be achieved with this library:
GitHub: https://github.com/tyrotoxin/AsyncEnumerable
NuGet.org: https://www.nuget.org/packages/AsyncEnumerator/
It's much simpler than Rx.

using System.Collections.Async;

static IAsyncEnumerable<string> ProduceItems(string[] urls)
{
  return new AsyncEnumerable<string>(async yield => {
    foreach (var url in urls) {
      var html = await UrlString.DownLoadHtmlAsync(url);
      await yield.ReturnAsync(html);
    }
  });
}

static async Task ConsumeItemsAsync(string[] urls)
{
  await ProduceItems(urls).ForEachAsync(async html => {
    await Console.Out.WriteLineAsync(html);
  });
}

This feature will be available as of C# 8.0. https://blogs.msdn.microsoft.com/dotnet/2018/11/12/building-c-8-0/

From MSDN

Async streams

The async/await feature of C# 5.0 lets you consume (and produce) asynchronous results in straightforward code, without callbacks:

async Task<int> GetBigResultAsync()
{
    var result = await GetResultAsync();
    if (result > 20) return result; 
    else return -1;
}

It is not so helpful if you want to consume (or produce) continuous streams of results, such as you might get from an IoT device or a cloud service. Async streams are there for that.

We introduce IAsyncEnumerable, which is exactly what you’d expect; an asynchronous version of IEnumerable. The language lets you await foreach over these to consume their elements, and yield return to them to produce elements.

async IAsyncEnumerable<int> GetBigResultsAsync()
{
    await foreach (var result in GetResultsAsync())
    {
        if (result > 20) yield return result; 
    }
}

There was a plan to do

https://github.com/dotnet/csharplang/issues/43

But currently it not possible

First of all, keep in mind that the Async stuff is not finished. The C# team still has a long way to go before C# 5 is released.

That being said, I think you may want to gather the tasks that are being fired off in the DownloadAllHtml function in a different way.

For example, you can use something like this:

IEnumerable<Task<string>> DownloadAllUrl(string[] urls)
{
    foreach(var url in urls)
    {
        yield return DownloadHtmlAsync(url);
    }
}

async Task<string> DownloadHtmlAsync(url)
{
    // Do your downloading here...
}

Not that the DownloadAllUrl function is NOT an async call. But, you can have the async call implemented on another function (i.e. DownloadHtmlAsync).

The Task Parallel Library has the .ContinueWhenAny and .ContinueWhenAll functions.

That can be used like this:

var tasks = DownloadAllUrl(...);
var tasksArray = tasks.ToArray();
var continuation = Task.Factory.ContinueWhenAll(tasksArray, completedTasks =>
{
    completedtask
});
continuation.RunSynchronously();