Task.Run isn't running asynchronously like I had thought it would?

Question

I have an import operation I'd like to execute on another thread, desiring the UI to respond right away. So, I started down the path and created an action like this:

[HttpPost]
public async Task<RedirectToRouteResult> ImportAsync(HttpPostedFileBase file)
{
    var importsRoot = Server.MapPath("~/App_Data/Imports");
    var path = Path.ChangeExtension(Path.Combine(importsRoot, Guid.NewGuid().ToString()), "txt");

    if (!Directory.Exists(importsRoot))
    {
        Directory.CreateDirectory(importsRoot);
    }

    file.SaveAs(path);

    // start the import process
    await ImportManager.Instance.StartImport(User.Identity.Name);

    return RedirectToAction("Index");
}

The ImportManager now serves two purposes:

1. Handle the import.
2. Notify all clients of the status of the import via SignalR.

The StartImport method looks like this:

public async Task StartImport(string user)
{
    string[] files;

    var importRoot = HttpContext.Current.Server.MapPath("~/App_Data/Imports");
    var processingRoot = HttpContext.Current.Server.MapPath("~/App_Data/Processing");
    var processedRoot = HttpContext.Current.Server.MapPath("~/App_Data/Processed");

    lock (lockObj)
    {
        //  make sure the "Processing" folder exists
        if (!Directory.Exists(processingRoot))
        {
            Directory.CreateDirectory(processingRoot);
        }

        //  find all of the files available and move them to the "Processing" folder
        files = Directory.GetFiles(importRoot);
        foreach (var file in files)
        {
            var fileName = Path.GetFileName(file);
            if (fileName == null)
            {
                continue;
            }

            File.Move(file, Path.Combine(processingRoot, fileName));
        }

        //  make sure the "Processed" directory exists
        if (!Directory.Exists(processedRoot))
        {
            Directory.CreateDirectory(processedRoot);
        }
    }

    await Task.Run(() =>
    {
        //  start processing the files
        foreach (var file in files)
        {
            var fileName = Path.GetFileName(file);
            if (fileName == null)
            {
                continue;
            }

            var processingFileName = Path.Combine(processingRoot, fileName);
            var processedFileName = Path.Combine(processedRoot, fileName);

            var recognizer = new Recognizer(processingFileName);
            recognizer.ProgressChanged += (s, e) => Clients.All.updateImportStatus(e.ProgressPercentage, user);
            recognizer.Recognize(DataManager.GetExclusionPatterns());

            //  move the file to the "Processed" folder
            File.Move(processingFileName, processedFileName);
        }

        Clients.All.importComplete();
    });
}

What's Happening?

Debugging we find that when I hit the await Task.Run(() => it runs synchronously (for a while anyway) because the UI doesn't get the request to redirect to Index until say 30K+ lines have been read.

How can I get this to simply execute and forget it? Do I need to use a different approach?

Answer 1

It is running asynchronously; but you are awaiting it:

await ImportManager.Instance.StartImport(User.Identity.Name);

return RedirectToAction("Index");

Not waiting; awaiting. The RedirectToAction is now a continuation that will be invoked when the other code has been completed.

If you don't want to await; don't await. However, you should think about what happens if an error occurs, etc. Do not let your async method raise an exception if nobody is going to observe it: bad things.

Answer 2

How can I get this to simply execute and forget it?

I strongly recommend that you not use "fire and forget" on ASP.NET. The core reason is because ASP.NET manages your application lifetime around request lifetimes. So if you have code executing that is not part of a request, then ASP.NET may take down your application. In the general case, this means that you cannot depend on that code actually executing.

Do I need to use a different approach?

The proper solution is to put the work into a reliable queue (e.g., Azure queue or MSMQ) and have an independent backend do the actual processing (e.g., Azure webrole, Azure worker role, or Win32 service). That's a fair amount of work, but it's the only reliable solution.

However, if you want to live dangerously and keep the work in-memory in your ASP.NET process, you should register the work with the ASP.NET runtime. You still don't have a guarantee that the processing will execute, but it's more likely, since ASP.NET is aware of your code. I have a NuGet package that will do that for you; just use BackgroundTaskManager.Run instead of Task.Run.