Unexpected behavior with await inside a ContinueWith block

Question

I have a slightly complex requirement of performing some tasks in parallel, and having to wait for some of them to finish before continuing. Now, I am encountering unexpected behavior, when I have a number of tasks, that I want executed in parallel, but inside a ContinueWith handler. I have whipped up a small sample to illustrate the problem:

var task1 = Task.Factory.StartNew(() =>
{
    Console.WriteLine("11");
    Thread.Sleep(1000);
    Console.WriteLine("12");
}).ContinueWith(async t =>
{
    Console.WriteLine("13");
    var innerTasks = new List<Task>();
    for (var i = 0; i < 10; i++)
    {
        var j = i;
        innerTasks.Add(Task.Factory.StartNew(() =>
        {
            Console.WriteLine("1_" + j + "_1");
            Thread.Sleep(500);
            Console.WriteLine("1_" + j + "_2");
        }));
    }
    await Task.WhenAll(innerTasks.ToArray());
    //Task.WaitAll(innerTasks.ToArray());
    Thread.Sleep(1000);
    Console.WriteLine("14");
});
var task2 = Task.Factory.StartNew(() =>
{
    Console.WriteLine("21");
    Thread.Sleep(1000);
    Console.WriteLine("22");
}).ContinueWith(t =>
{
    Console.WriteLine("23");
    Thread.Sleep(1000);
    Console.WriteLine("24");
});
Console.WriteLine("1");
await Task.WhenAll(task1, task2);
Console.WriteLine("2");

The basic pattern is: - Task 1 should be executed in parallel with Task 2. - Once the first part of part 1 is done, it should do some more things in parallel. I want to complete, once everything is done.

I expect the following result:

1 <- Start
11 / 21 <- The initial task start
12 / 22 <- The initial task end
13 / 23 <- The continuation task start
Some combinations of "1_[0..9]_[1..2]" and 24 <- the "inner" tasks of task 1 + the continuation of task 2 end
14 <- The end of the task 1 continuation
2 <- The end

Instead, what happens, is that the await Task.WhenAll(innerTasks.ToArray()); does not "block" the continuation task from completing. So, the inner tasks execute after the outer await Task.WhenAll(task1, task2); has completed. The result is something like:

1 <- Start
11 / 21 <- The initial task start
12 / 22 <- The initial task end
13 / 23 <- The continuation task start
Some combinations of "1_[0..9]_[1..2]" and 24 <- the "inner" tasks of task 1 + the continuation of task 2 end
2 <- The end
Some more combinations of "1_[0..9]_[1..2]" <- the "inner" tasks of task 1
14 <- The end of the task 1 continuation

If, instead, I use Task.WaitAll(innerTasks.ToArray()), everything seems to work as expected. Of course, I would not want to use WaitAll, so I won't block any threads.

My questions are:

1. Why is this unexpected behavior occuring?
2. How can I remedy the situation without blocking any threads?

Thanks a lot in advance for any pointers!

Answer 1

As noted in the comments, what you're seeing is normal. The Task returned by ContinueWith() completes when the delegate passed to and invoked by ContinueWith() finishes executing. This happens the first time the anonymous method uses the await statement, and the delegate returns a Task object itself that represents the eventual completion of the entire anonymous method.

Since you are only waiting on the ContinueWith() task, and this task only represents the availability of the task that represents the anonymous method, not the completion of that task, your code doesn't wait.

From your example, it's not clear what the best fix is. But if you make this small change, it will do what you want:

await Task.WhenAll(await task1, task2);

I.e. in the WhenAll() call, don't wait on the ContinueWith() task itself, but rather on the task that task will eventually return. Use await here to avoid blocking the thread while you wait for that task to be available.

Answer 2

You're using the wrong tools. Instead of StartNew, use Task.Run. Instead of ContinueWith, use await:

var task1 = Task1();
var task2 = Task2();
Console.WriteLine("1");
await Task.WhenAll(task1, task2);
Console.WriteLine("2");

private async Task Task1()
{
  await Task.Run(() =>
  {
    Console.WriteLine("11");
    Thread.Sleep(1000);
    Console.WriteLine("12");
  });
  Console.WriteLine("13");
  var innerTasks = new List<Task>();
  for (var i = 0; i < 10; i++)
  {
    innerTasks.Add(Task.Run(() =>
    {
      Console.WriteLine("1_" + i + "_1");
      Thread.Sleep(500);
      Console.WriteLine("1_" + i + "_2");
    }));
    await Task.WhenAll(innerTasks);
  }
  Thread.Sleep(1000);
  Console.WriteLine("14");
}

private async Task Task2()
{
  await Task.Run(() =>
  {
    Console.WriteLine("21");
    Thread.Sleep(1000);
    Console.WriteLine("22");
  });
  Console.WriteLine("23");
  Thread.Sleep(1000);
  Console.WriteLine("24");
}

Task.Run and await are superior here because they correct a lot of unexpected behavior in StartNew/ContinueWith. In particular, asynchronous delegates and (for Task.Run) always using the thread pool.

I have more detailed info on my blog regarding why you shouldn't use StartNew and why you shouldn't use ContinueWith.

Answer 3

When using async methods/lambdas with StartNew, you either wait on the returned task and the contained task:

var task = Task.Factory.StartNew(async () => { /* ... */ });
task.Wait();
task.Result.Wait();
// consume task.Result.Result

Or you use the extension method Unwrap on the result of StartNew and wait on the task it returns.

var task = Task.Factory.StartNew(async () => { /* ... */ })
    .Unwrap();
task.Wait();
// consume task.Result

---

The following discussion goes along the line that Task.Factory.StartNew and ContinueWith should be avoided in specific cases, such as when you don't provide creation or continuation options or when you don't provide a task scheduler.

I don't agree that Task.Factory.StartNew shouldn't be used, I agree that you should use (or consider using) Task.Run wherever you use a Task.Factory.StartNew method overload that doesn't take TaskCreationOptions or a TaskScheduler.

Note that this only applies to the default Task.Factory. I've used custom task factories where I chose to use the StartNew overloads without options and task scheduler, because I configured the factories specific defaults for my needs.

Likewise, I don't agree that ContinueWith shouldn't be used, I agree that you should use (or consider using) async/await wherever you use a ContinueWith method overload that doesn't take TaskContinuationOptions or a TaskScheduler.

For instance, up to C# 5, the most practical way to workaround the limitation of await not being supported in catch and finally blocks is to use ContinueWith.

C# 6:

try
{
    return await something;
}
catch (SpecificException ex)
{
    await somethingElse;
    // throw;
}
finally
{
    await cleanup;
}

Equivalent before C# 6:

return await something
    .ContinueWith(async somethingTask =>
    {
        var ex = somethingTask.Exception.InnerException as SpecificException;
        if (ex != null)
        {
            await somethingElse;
            // await somethingTask;
        }
    },
        CancellationToken.None,
        TaskContinuationOptions.DenyChildAttach | TaskContinuationOptions.NotOnRanToCompletion,
        TaskScheduler.Default)
    .Unwrap()
    .ContinueWith(async catchTask =>
    {
        await cleanup;
        await catchTask;
    },
        CancellationToken.None,
        TaskContinuationOptions.DenyChildAttach,
        TaskScheduler.Default)
    .Unwrap();

Since, as I told, in some cases I have a TaskFactory with specific defaults, I've defined a few extension methods that take a TaskFactory, reducing the error chance of not passing one of the arguments (I known I can always forget to pass the factory itself):

public static Task ContinueWhen(this TaskFactory taskFactory, Task task, Action<Task> continuationAction)
{
    return task.ContinueWith(continuationAction, taskFactory.CancellationToken, taskFactory.ContinuationOptions, taskFactory.Scheduler);
}

public static Task<TResult> ContinueWhen<TResult>(this TaskFactory taskFactory, Task task, Func<Task, TResult> continuationFunction)
{
    return task.ContinueWith(continuationFunction, taskFactory.CancellationToken, taskFactory.ContinuationOptions, taskFactory.Scheduler);
}

// Repeat with argument combinations:
// - Task<TResult> task (instead of non-generic Task task)
// - object state
// - bool notOnRanToCompletion (useful in C# before 6)

Usage:

// using namespace that contains static task extensions class
var task = taskFactory.ContinueWhen(existsingTask, t => Continue(a, b, c));
var asyncTask = taskFactory.ContinueWhen(existingTask, async t => await ContinueAsync(a, b, c))
    .Unwrap();

I decided not to mimic Task.Run by not overloading the same method name to unwrapping task-returning delegates, it's really not always what you want. Actually, I didn't even implement ContinueWhenAsync extension methods so you need to use Unwrap or two awaits.

Often, these continuations are I/O asynchronous operations, and the pre- and post-processing overhead should be so small that you shouldn't care if it starts running synchronously up to the first yielding point, or even if it completes synchronously (e.g. using an underlying MemoryStream or a mocked DB access). Also, most of them don't depend on a synchronization context.

Whenever you apply the Unwrap extension method or two awaits, you should check if the task falls in this category. If so, async/await is most probably a better choice than starting a task.

For asynchronous operations with a non-negligible synchronous overhead, starting a new task may be preferable. Even so, a notable exception where async/await is still a better choice is if your code is async from the start, such as an async method invoked by a framework or host (ASP.NET, WCF, NServiceBus 6+, etc.), as the overhead is your actual business. For long processing, you may consider using Task.Yield with care. One of the tenets of asynchronous code is to not be too fine grained, however, too coarse grained is just as bad: a set of heavy-duty tasks may prevent the processing of queued lightweight tasks.

If the asynchronous operation depends on a synchronization context, you can still use async/await if you're within that context (in this case, think twice or more before using .ConfigureAwait(false)), otherwise, start a new task using a task scheduler from the respective synchronization context.