How to "trick" Azure Function into running longer than 10 minutes

Question

Azure Functions have a time limit of 10 minutes. Suppose I have a long-running task such as downloading a file that takes 1 hr to download.

[FunctionName("PerformDownload")]
[return: Queue("download-path-queue")]
public static async Task<string> RunAsync([QueueTrigger("download-url-queue")] string url, TraceWriter log)
{
   string downloadPath = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString);
   log.Info($"Downloading file at url {url} to {downloadPath} ...");
   using (var client = new WebClient())
   {
       await client.DownloadFileAsync(new Uri(url), myLocalFilePath);
   }
   log.Info("Finished!");   
}

Is there any hacky way to make something like this start and then resume in another function before the time limit expires? Or is there a better way altogether to integrate some long task like this into a workflow that uses Azure Functions?

(On a slightly related note, is plain Azure Web Jobs obsolete? I can't find it under Resources.)

Answer 1

Adding for others who might come across this post: Workflows composed of several Azure Functions can be created in code using the Durable Functions extension, which can be used to create orchestration functions that schedule async tasks, shut down, and are reawakened when said async work is complete.

They're not a direct solution for long-running tasks that require an open TCP port, such as downloading a file, (for that, a function running on an App Service Plan has no execution time limit), but it can be used to integrate such tasks into a larger workflow.

Answer 2

Is there any hacky way to make something like this start and then resume in another function before the time limit expires?

If you are on a Consumption Plan you have no control over how long your Function App runs, and so it would not be reliable to use background threads that continue running after your Function entry point completes.

On an App Service plan you're running on VMs you pay for, so you can configure your Function App to run continuously. Also AFAIK you don't have to have a Function timeout on an App Service Plan, so your main Function entry point can run for as long as you want.

Or is there a better way altogether to integrate some long task like this into a workflow that uses Azure Functions?

Yes. Use Azure Data Factory to copy data into Blob Storage, and then process it. The Data Factory pipeline can call Functions both before and after the copy activity.