How to use non-thread-safe async/await APIs and patterns with ASP.NET Web API?

Question

This question has been triggered by EF Data Context - Async/Await & Multithreading. I've answered that one, but haven't provided any ultimate solution.

The original problem is that there are a lot of useful .NET APIs out there (like Microsoft Entity Framework's DbContext), which provide asynchronous methods designed to be used with await, yet they are documented as not thread-safe. That makes them great for use in desktop UI apps, but not for server-side apps. [EDITED] This might not actually apply to DbContext, here is Microsoft's statement on EF6 thread safety, judge for yourself. [/EDITED]

There are also some established code patterns falling into the same category, like calling a WCF service proxy with OperationContextScope (asked here and here), e.g.:

using (var docClient = CreateDocumentServiceClient())
using (new OperationContextScope(docClient.InnerChannel))
{
    return await docClient.GetDocumentAsync(docId);
}

This may fail because OperationContextScope uses thread local storage in its implementation.

The source of the problem is AspNetSynchronizationContext which is used in asynchronous ASP.NET pages to fulfill more HTTP requests with less threads from ASP.NET thread pool. With AspNetSynchronizationContext, an await continuation can be queued on a different thread from the one which initiated the async operation, while the original thread is released to the pool and can be used to serve another HTTP request. This substantially improves the server-side code scalability. The mechanism is described in great details in It's All About the SynchronizationContext, a must-read. So, while there is no concurrent API access involved, a potential thread switch still prevents us from using the aforementioned APIs.

I've been thinking about how to solve this without sacrificing the scalability. Apparently, the only way to have those APIs back is to maintain thread affinity for the scope of the async calls potentially affected by a thread switch.

Let's say we have such thread affinity. Most of those calls are IO-bound anyway (There Is No Thread). While an async task is pending, the thread it's been originated on can be used to serve a continuation of another similar task, which result is already available. Thus, it shouldn't hurt scalability too much. This approach is nothing new, in fact, a similar single-threaded model is successfully used by Node.js. IMO, this is one of those things that make Node.js so popular.

I don't see why this approach could not be used in ASP.NET context. A custom task scheduler (let's call it ThreadAffinityTaskScheduler) might maintain a separate pool of "affinity apartment" threads, to improve scalability even further. Once the task has been queued to one of those "apartment" threads, all await continuations inside the task will be taking place on the very same thread.

Here's how a non-thread-safe API from the linked question might be used with such ThreadAffinityTaskScheduler:

// create a global instance of ThreadAffinityTaskScheduler - per web app
public static class GlobalState 
{
    public static ThreadAffinityTaskScheduler TaScheduler { get; private set; }

    public static GlobalState 
    {
        GlobalState.TaScheduler = new ThreadAffinityTaskScheduler(
            numberOfThreads: 10);
    }
}

// ...

// run a task which uses non-thread-safe APIs
var result = await GlobalState.TaScheduler.Run(() => 
{
    using (var dataContext = new DataContext())
    {
        var something = await dataContext.someEntities.FirstOrDefaultAsync(e => e.Id == 1);
        var morething = await dataContext.someEntities.FirstOrDefaultAsync(e => e.Id == 2);
        // ... 
        // transform "something" and "morething" into thread-safe objects and return the result
        return data;
    }
}, CancellationToken.None);

I went ahead and implemented ThreadAffinityTaskScheduler as a proof of concept, based on the Stephen Toub's excellent StaTaskScheduler. The pool threads maintained by ThreadAffinityTaskScheduler are not STA thread in the classic COM sense, but they do implement thread affinity for await continuations (SingleThreadSynchronizationContext is responsible for that).

So far, I've tested this code as console app and it appears to work as designed. I haven't tested it inside an ASP.NET page yet. I don't have a lot of production ASP.NET development experience, so my questions are:

1. Does it make sense to use this approach over simple synchronous invocation of non-thread-safe APIs in ASP.NET (the main goal is to avoid sacrificing scalability)?

2. Is there alternative approaches, besides using synchronous API invocations or avoiding those APis at all?

3. Has anyone used something similar in ASP.NET MVC or Web API projects and is ready to share his/her experience?

4. Any advice on how to stress-test and profile this approach with ASP.NET would be appreciated.

Answer 1

Entity Framework will (should) handle thread jumps across await points just fine; if it doesn't, then that's a bug in EF. OTOH, OperationContextScope is based on TLS and is not await-safe.

1. Synchronous APIs maintain your ASP.NET context; this includes things such as user identity and culture that are often important during processing. Also, a number of ASP.NET APIs assume they are running on an actual ASP.NET context (I don't mean just using HttpContext.Current; I mean actually assuming that SynchronizationContext.Current is an instance of AspNetSynchronizationContext).

2-3. I have used my own single-threaded context nested directly within the ASP.NET context, in attempts to get async MVC child actions working without having to duplicate code. However, not only do you lose the scalability benefits (for that part of the request, at least), you also run into the ASP.NET APIs assuming that they're running on an ASP.NET context.

So, I have never used this approach in production. I just end up using the synchronous APIs when necessary.

Answer 2

You should not intertwine multithreading with asynchrony. The problem with an object not being thread-safe is when a single instance (or static) is accessed by multiple threads at the same time. With async calls the context is possibly accessed from a different thread in the continuation, but never at the same time (when not shared across multiple requests, but that isn't good in the first place).