Async lambda to Expression<Func<Task>>

Question

It is widely known that I can convert ordinary lambda expression to Expression<T>:

Func<int> foo1 = () => 0; // delegate compiles fine
Expression<Func<int>> foo2 = () => 0; // expression compiles fine

How could I do the same with async lambda? I've tried the following analogy:

Func<Task<int>> bar1 = async () => 0; // also compiles (async lambda example)
Expression<Func<Task<int>>> bar2 = async () => 0; // CS1989: Async lambda expressions cannot be converted to expression trees

Is there any workaround possible?

Answer 1

It is indeed possible to implement async expression trees, but there is no framework support (yet?) for building async expression trees. Therefore this is definitively not a simple undertaking, but I have several implementations in everyday productive use.

The ingredients needed are the following:

• A helper class derived from TaskCompletionSource which is used to provide the task and all the required stuff related to it.

  We need to add a State property (you can use a different name, but this aligns it with the helpers generated by the C# compiler for async-await) which keeps track of which state the state machine is currently at.

  Then we need to have a MoveNext property which is a Action. This will be called to work on the next state of the state machine.

  Finally we need a place to store the currently pending Awaiter, which would be a property of type object.

  The async method terminates by either using SetResult, SetException (or SetCanceled).

  Such an implementation could look like this:

internal class AsyncExpressionContext<T>: TaskCompletionSource<T> {
    public int State {
        get;
        set;
    }

    public object Awaiter {
        get;
        set;
    }

    public Action MoveNext {
        get;
    }

    public AsyncExpressionContext(Action<AsyncExpressionContext<T>> stateMachineFunc): base(TaskCreationOptions.RunContinuationsAsynchronously) {
        MoveNext = delegate {
            try {
                stateMachineFunc(this);
            }
            catch (Exception ex) {
                State = -1;
                Awaiter = null;
                SetException(ex);
            }
        };
    }
}

• A state machine lambda expression which implements the actual state machine as switch statement, something like that (does not compile as-is, but should give an idea of what needs to be done):

var paraContext = Expression.Parameter(AsyncExpressionContext<T>, "context");
var stateMachineLambda = Expression.Lambda<Action<AsyncExpressionContext<T>>>(Expression.Block(new[] { varGlobal },
    Expression.Switch(typeof(void),
        Expression.Property(paraContext, nameof(AsyncExpressionContext<T>.State)),
        Expression.Throw(
            Expression.New(ctor_InvalidOperationException, Expression.Constant("Invalid state"))),
    null,
    stateMachineCases));

Each of the cases does implement one state of the state machine. I'm not going to get much into the details of the async-await state machine concept in general since there are excellent resources available, especially many blog posts, which explain everything in great detail.

https://devblogs.microsoft.com/premier-developer/dissecting-the-async-methods-in-c/

By leveraging labels and goto expressions (which can jump across blocks if they do not carry a value) it is possible to implement the "hot path optimization" when async methods return synchronously after having been called.

The basic concept goes like this (pseudicode):

State 0 (start state):
  - Initiate async call, which returns an awaitable object.
  - Optionally and if present call ConfigureAwait(false) to get another awaiter.
  - Check the IsCompleted property of the awaiter.
    - If true, call GetResult() on the awaiter and store the the result in a "global" variable, then jump to the label "state0continuation"
    - If false, store the awaiter and the next state in the context object, then call OnCompleted(context.MoveNext) on the awaiter and return

State X (continuation states):
  - Cast the awaiter from the context object back to its original type and call GetResult(), store its result in the same "global" variable.
  - Label "state0continuation" goes here; if the call was synchronous we already have our value in the "global" variable
  - Do some non-async work
  - To end the async call, call SetResult() on the context and return (setting the state property to an invalid value and clearing the awaiter property may be a good idea for keeping things tidy)
  - You can make other async calls just as shown in state 0 and move to other states

• A "bootstrapper" expression which creates the TaskCompletionSource and starts the state machine. This is what will be exposed as async lambda. You can, of course, also add parameters and either pass them through closure or by adding them to the context object.

var varContext = Expression.Variable(typeof(AsyncExpressionContext<T>), "context");
var asyncLambda = Expression.Lambda<Func<Task<T>>>(
    Expression.Block(
        Expression.Assign(
            varContext,
            Expression.New(ctor_AsyncExpressionContext,
                Expression.Lambda<Action<AsyncExpressionContext<T>>>(
                    stateMachineExression,
                    paraContext))),
    Expression.Invoke(
        Expression.Property(varContext, nameof(AsyncExpressionContext<T>.MoveNext)),
        varContext),
    Expression.Property(varContext, nameof(AsyncExpressionContext<T>.Task)));

This is pretty much all which is needed for a linear async method. If you want to add conditional branches, things get a bit trickier in order to get the flow to the next state correct, but it is just as well possible and working.