Async Await Recursion in .NET 4.8 causes StackoverflowException (not in .Net Core 3.1!)

Question

Why does the following code cause a StackOverflowException in .Net4.8 with only a 17-depth recursion? However this does not happen in NetCore 3.1 (I can set the count to 10_000 and it still works)

class Program
{
  static async Task Main(string[] args)
  {
    try
    {
      await TestAsync(17);
    }
    catch(Exception e)
    {
      Console.WriteLine("Exception caught: " + e);
    }
  }

  static async Task TestAsync(int count)
  {
    await Task.Run(() =>
    {
      if (count <= 0)
        throw new Exception("ex");
    });

    Console.WriteLine(count);
    await TestAsync2(count);
  }

  static async Task TestAsync2(int count) => await TestAsync3(count);
  static async Task TestAsync3(int count) => await TestAsync4(count);
  static async Task TestAsync4(int count) => await TestAsync5(count);
  static async Task TestAsync5(int count) => await TestAsync6(count);
  static async Task TestAsync6(int count) => await TestAsync(count - 1);
}

Is this a known bug in .Net 4.8? I would except a lot more than 17 levels of recursion in such a function... Does this effectively mean writing recursions with async/await is not recommended?

Update: Simplified version

class Program
{
  // needs to be compiled as AnyCpu Prefer 64-bit
  static async Task Main(string[] args)
  {
    try
    {
      await TestAsync(97); // 96 still works
    }
    catch(Exception e)
    {
      Console.WriteLine("Exception caught: " + e);
    }
  }

  static async Task TestAsync(int count)
  {
    await Task.Run(() =>
    {
      if (count <= 0)
        throw new Exception("ex");
    });

    Console.WriteLine(count);
    await TestAsync(count-1);
  }
}

It only happens so fast when choosing Any Cpu with Prefer 32-bit disabled, but is reproducable on multiple machines (Windows 1903 and 1909) on multiple .net versions (.Net 4.7.2 and .Net 4.8)

Answer 1

I suspect you're seeing the Stack Overflow on the completions - i.e., every number is printed out all the way down to 1 before the Stack Overflow message.

My guess is that this behavior is because await uses synchronous continuations. There's supposed to be code that prevents synchronous continuations from overflowing the stack, but it's heuristic and doesn't always work.

I suspect this behavior doesn't happen on .NET Core because a lot of optimization work has gone into .NET Core's async support, likely meaning that continuations on that platform take up less stack space, making the heuristic check work. It is also possible that the heuristic itself was fixed in .NET Core. Either way, I wouldn't hold my breath expecting .NET Framework to get those updates.

I would except a lot more than 17 levels of recursion in such a function...

Not really 17. You've got 102 levels of recursion (17 * 6). To measure the actual stack space taken up, it would be 17 * 6 * (number of stacks to resume continuations). On my machine, 17 works; it fails somewhere over 200 (1200 calls deep).

Bear in mind that this only happens for long sequences of tail recursive asynchronous functions - i.e., none of them have any more asynchronous work to do after their await. If you change any of the functions to have some other asynchronous work after their recursive await, that will avoid the stack overflow:

static async Task TestAsync(int count)
{
  await Task.Run(() =>
  {
    if (count <= 0)
      throw new Exception("ex");
  });

  Console.WriteLine(count);
  try
  {
    await TestAsync2(count);
  }
  finally
  {
    await Task.Yield(); // some other async work
  }
}