Must async methods be supported by OS or is async program level feature

Question

Tutorials sometimes point implementation of own async methods as for example this code:

async public static Task GetHttpResponseAsync()
{
   using (HttpClient httpClient = new HttpClient())
   {
       HttpResponseMessage response = await httpClient.GetAsync(...);
       Console.WriteLine(response.Something);
   }            
}

what is clear to me is how async work generally, but none of tutorials explains how internally are implemented

httpClient.GetAsync(...);

which is really important to understand how asynchoronus code works in details. What make me curious is if internall operations of GetAsync (those method or other async method) are registered in some kind of container where this code is executed? Are async methods must be supported by operating system (f.e it uses windows api)? If I would like to implement my own asynchronous file downloader (from disk and without essential part from .NET framework), how would I implement it, should I register my method somewhere for further invocation?

It's pretty clear for me that internally, compiler makes state machine and after DoSomething() method do what it have to, it just invoke this state machine again to resume executing code after await.

Also what is unclear for me is that how async code can run on same thread. I think that maintaining state machine must be on the same thread but how the code from httpClient.GetAsync() can be run on the same thread and doesn't interrupt other operations (f.e gui). There must be something that make this code runs on separate thread (in all cases). Am I wrong? What I missed?

Additional explanation of my question: In JavaScript, as far as I know and understand, async methods works by registering them in some kind of container (which runs them one by one on separate thread) which executes this method. After execution of the method complete, result is returned to user context, It's clear for me, Is that work in the same way here?

Answer 1

In short, true asynchrony must be provided at the OS level. As you've noted, async/await is a language level feature that uses compiler generated state machines to "break up" your method into pieces that can run asynchronously, but it relies on OS primitives (interrupts, threads) to actually perform this work in an asynchronous manner.

However, it's important to note that there will often not be a thread created to handle your async operation. I'll defer to this expertly-written article to describe why this is the case: http://blog.stephencleary.com/2013/11/there-is-no-thread.html

On Windows, the primary mechanism for performing asynchronous work is with I/O Completion Ports. This is a Windows API that is used under the hood by many .NET types, including the HttpClient you're using.

Note that for non-I/O operations, you can also always use Threads, or better yet, the Thread Pool API, to perform background work that will complete asynchronously.

The ReadFile (or newer ReadFileEx) function in the Windows API is designed to work with async I/O. When you call ReadFile, you can use the FILE_FLAG_OVERLAPPED flag and pass an OVERLAPPED structure to the lpOverlapped argument, which enables async reads. I would encourage you to use this API to design your file downloader.

---

In summary:

• There will not always be a thread created for async operations.
• async/await is a language feature, but relies on various Windows APIs to achieve true asynchrony.
• If the work you are doing is I/O bound, consider using Asynchronous I/O: https://msdn.microsoft.com/en-us/library/windows/desktop/aa365683(v=vs.85).aspx
• If the work you are doing is CPU bound, consider using Threading: https://msdn.microsoft.com/en-us/library/windows/desktop/ms684841(v=vs.85).aspx
• Now that .NET Core is open source on GitHub, you can actually inspect the source code to see what's going on under the hood. Here is HttpClient.cs, which uses HttpMessageHandler.cs -> HttpClientHandler.Windows.cs -> WinHttpHandler.cs -> Interop.winhttp.cs -> PInvoke into the WinHTTP API native DLL -> Winsock sockets with an I/O completion port.