Azure Functions ( Http Triggered) started throwing TaskCancelled Exception randomly

Question

How do I go about troubleshooting this?

• I have a couple of Azure functions that use the following trigger signature [HttpTrigger(AuthorizationLevel.Anonymous, "get", "post").

• The functions are running on a consumption plan.

• They get triggered 2-3 times a day.

The function intermittently fails to trigger with the following exception:

Diagnostic from Azure shows that the task was canceled before the worker process was called.

public class AlpacaSample 
{
    private ILogger logger;

    public AlpacaSample(ILoggerFactory loggerFactory)
    {
        this.logger = loggerFactory.CreateLogger<AlpacaSample>();
    }

    [Function("AlpacaSample")]
    public async Task<HttpResponseData> Run([HttpTrigger(AuthorizationLevel.Anonymous, "get", "post")] HttpRequestData req)
    {
        try
        {
            this.requestData = req;
            string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
            var message = $"Received a trigger {requestBody}";
            this.logger.LogInformation(message);

            // Business logic removed for brevity 


            return await this.CreateStringResponse(req, "All Good");
        }

        catch (Exception ex)
        {
            string responseMessage =
                   $"Error occurrend while handling the message. Error: {ex.Message} \n {ex}";
            this.logger.LogError(responseMessage);
            return await this.CreateStringResponse(req, responseMessage);
        }


    }

    private async Task<HttpResponseData> CreateStringResponse(HttpRequestData req, string msg, HttpStatusCode httpStatusCode = HttpStatusCode.OK)
    {
        var res = HttpResponseData.CreateResponse(req);
        res.StatusCode = httpStatusCode;
        await res.WriteStringAsync(msg);
        return res;
    }
}

Full Call stack :

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor+<ExecuteWithLoggingAsync>d__26.MoveNext (Microsoft.Azure.WebJobs.Host, Version=3.0.37.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: D:\a\_work\1\s\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:352)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor+<TryExecuteAsync>d__18.MoveNext (Microsoft.Azure.WebJobs.Host, Version=3.0.37.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: D:\a\_work\1\s\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:108)
Inner exception Microsoft.Azure.WebJobs.Script.Description.FunctionInvocationCanceledException handled at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw:
   at Microsoft.Azure.WebJobs.Script.Description.WorkerFunctionInvoker+<InvokeCore>d__9.MoveNext (Microsoft.Azure.WebJobs.Script, Version=4.21.0.0, Culture=neutral, PublicKeyToken=null: /_/src/WebJobs.Script/Description/Workers/WorkerFunctionInvoker.cs:106)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.Azure.WebJobs.Script.Description.FunctionInvokerBase+<Invoke>d__24.MoveNext (Microsoft.Azure.WebJobs.Script, Version=4.21.0.0, Culture=neutral, PublicKeyToken=null: /_/src/WebJobs.Script/Description/FunctionInvokerBase.cs:82)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.Azure.WebJobs.Script.Description.FunctionGenerator+<Coerce>d__3`1.MoveNext (Microsoft.Azure.WebJobs.Script, Version=4.21.0.0, Culture=neutral, PublicKeyToken=null: /_/src/WebJobs.Script/Description/FunctionGenerator.cs:225)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.Azure.WebJobs.Host.Executors.FunctionInvoker`2+<InvokeAsync>d__10.MoveNext (Microsoft.Azure.WebJobs.Host, Version=3.0.37.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: D:\a\_work\1\s\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionInvoker.cs:52)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor+<InvokeWithTimeoutAsync>d__33.MoveNext (Microsoft.Azure.WebJobs.Host, Version=3.0.37.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: D:\a\_work\1\s\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:581)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor+<ExecuteWithWatchersAsync>d__32.MoveNext (Microsoft.Azure.WebJobs.Host, Version=3.0.37.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: D:\a\_work\1\s\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:527)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor+<ExecuteWithLoggingAsync>d__26.MoveNext (Microsoft.Azure.WebJobs.Host, Version=3.0.37.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: D:\a\_work\1\s\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:306)
Inner exception System.Threading.Tasks.TaskCanceledException handled at Microsoft.Azure.WebJobs.Script.Description.WorkerFunctionInvoker+<InvokeCore>d__9.MoveNext:
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.Azure.WebJobs.Script.Description.WorkerFunctionInvoker+<InvokeCore>d__9.MoveNext (Microsoft.Azure.WebJobs.Script, Version=4.21.0.0, Culture=neutral, PublicKeyToken=null: /_/src/WebJobs.Script/Description/Workers/WorkerFunctionInvoker.cs:101)

Just My Code call stack

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor+<ExecuteWithLoggingAsync>d__26.MoveNext (Microsoft.Azure.WebJobs.Host, Version=3.0.37.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: D:\a\_work\1\s\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:352)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor+<TryExecuteAsync>d__18.MoveNext (Microsoft.Azure.WebJobs.Host, Version=3.0.37.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: D:\a\_work\1\s\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:108)
Inner exception Microsoft.Azure.WebJobs.Script.Description.FunctionInvocationCanceledException handled at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw:
   at Microsoft.Azure.WebJobs.Script.Description.WorkerFunctionInvoker+<InvokeCore>d__9.MoveNext (Microsoft.Azure.WebJobs.Script, Version=4.21.0.0, Culture=neutral, PublicKeyToken=null: /_/src/WebJobs.Script/Description/Workers/WorkerFunctionInvoker.cs:106)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.Azure.WebJobs.Script.Description.FunctionInvokerBase+<Invoke>d__24.MoveNext (Microsoft.Azure.WebJobs.Script, Version=4.21.0.0, Culture=neutral, PublicKeyToken=null: /_/src/WebJobs.Script/Description/FunctionInvokerBase.cs:82)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.Azure.WebJobs.Script.Description.FunctionGenerator+<Coerce>d__3`1.MoveNext (Microsoft.Azure.WebJobs.Script, Version=4.21.0.0, Culture=neutral, PublicKeyToken=null: /_/src/WebJobs.Script/Description/FunctionGenerator.cs:225)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.Azure.WebJobs.Host.Executors.FunctionInvoker`2+<InvokeAsync>d__10.MoveNext (Microsoft.Azure.WebJobs.Host, Version=3.0.37.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: D:\a\_work\1\s\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionInvoker.cs:52)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor+<InvokeWithTimeoutAsync>d__33.MoveNext (Microsoft.Azure.WebJobs.Host, Version=3.0.37.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: D:\a\_work\1\s\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:581)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor+<ExecuteWithWatchersAsync>d__32.MoveNext (Microsoft.Azure.WebJobs.Host, Version=3.0.37.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: D:\a\_work\1\s\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:527)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor+<ExecuteWithLoggingAsync>d__26.MoveNext (Microsoft.Azure.WebJobs.Host, Version=3.0.37.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: D:\a\_work\1\s\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:306)
Inner exception System.Threading.Tasks.TaskCanceledException handled at Microsoft.Azure.WebJobs.Script.Description.WorkerFunctionInvoker+<InvokeCore>d__9.MoveNext:
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.Azure.WebJobs.Script.Description.WorkerFunctionInvoker+<InvokeCore>d__9.MoveNext (Microsoft.Azure.WebJobs.Script, Version=4.21.0.0, Culture=neutral, PublicKeyToken=null: /_/src/WebJobs.Script/Description/Workers/WorkerFunctionInvoker.cs:101)

Trace

6/1/2023, 7:45:03 AM
-
REQUEST
URL: https://******.azurewebsites.net/api/******
Response code: 0
Response time: 3.1 s
6/1/2023, 7:45:03 AM
-
TRACE
FUNCTIONS_WORKER_RUNTIME set to dotnet-isolated. Skipping WorkerConfig for language: java
Severity level: Information
6/1/2023, 7:45:03 AM
-
TRACE
FUNCTIONS_WORKER_RUNTIME set to dotnet-isolated. Skipping WorkerConfig for language: node
Severity level: Information
6/1/2023, 7:45:03 AM
-
TRACE
FUNCTIONS_WORKER_RUNTIME set to dotnet-isolated. Skipping WorkerConfig for language: powershell
Severity level: Information
6/1/2023, 7:45:03 AM
-
TRACE
Initializing Warmup Extension.
Severity level: Information
6/1/2023, 7:45:03 AM
-
TRACE
Initializing Host. OperationId: 'fd2e77ef-c3b6-4a31-9d77-24c0de283e1f'.
Severity level: Information
6/1/2023, 7:45:03 AM
-
TRACE
Host initialization: ConsecutiveErrors=0, StartupCount=4, OperationId=fd2e77ef-c3b6-4a31-9d77-24c0de283e1f
Severity level: Information
6/1/2023, 7:45:03 AM
-
TRACE
Loading functions metadata
Severity level: Information
6/1/2023, 7:45:03 AM
-
TRACE
Reading functions metadata
Severity level: Information
6/1/2023, 7:45:04 AM
-
TRACE
2 functions found
Severity level: Information
6/1/2023, 7:45:04 AM
-
TRACE
1 functions loaded
Severity level: Information
6/1/2023, 7:45:04 AM
-
TRACE
ApplicationInsightsLoggerOptions { "SamplingSettings": { "EvaluationInterval": "00:00:15", "InitialSamplingPercentage": 100.0, "MaxSamplingPercentage": 100.0, "MaxTelemetryItemsPerSecond": 20.0, "MinSamplingPercentage": 0.1, "MovingAverageRatio": 0.25, "SamplingPercentageDecreaseTimeout": "00:02:00", "SamplingPercentageIncreaseTimeout": "00:15:00" }, "SamplingExcludedTypes": "Request", "SamplingIncludedTypes": null, "SnapshotConfiguration": null, "EnablePerformanceCountersCollection": true, "HttpAutoCollectionOptions": { "EnableHttpTriggerExtendedInfoCollection": true, "EnableW3CDistributedTracing": true, "EnableResponseHeaderInjection": true }, "LiveMetricsInitializationDelay": "00:00:15", "EnableLiveMetrics": true, "EnableDependencyTracking": true, "DependencyTrackingOptions": null }
Severity level: Information
6/1/2023, 7:45:04 AM
-
TRACE
LoggerFilterOptions { "MinLevel": "None", "Rules": [ { "ProviderName": null, "CategoryName": null, "LogLevel": null, "Filter": "<AddFilter>b__0" }, { "ProviderName": "Microsoft.Azure.WebJobs.Script.WebHost.Diagnostics.SystemLoggerProvider", "CategoryName": null, "LogLevel": "None", "Filter": null }, { "ProviderName": "Microsoft.Azure.WebJobs.Script.WebHost.Diagnostics.SystemLoggerProvider", "CategoryName": null, "LogLevel": null, "Filter": "<AddFilter>b__0" }, { "ProviderName": "Microsoft.Azure.WebJobs.Logging.ApplicationInsights.ApplicationInsightsLoggerProvider", "CategoryName": null, "LogLevel": "Trace", "Filter": null } ] }
Severity level: Information
6/1/2023, 7:45:04 AM
-
TRACE
LoggerFilterOptions { "MinLevel": "None", "Rules": [ { "ProviderName": null, "CategoryName": null, "LogLevel": null, "Filter": "<AddFilter>b__0" }, { "ProviderName": "Microsoft.Azure.WebJobs.Script.WebHost.Diagnostics.SystemLoggerProvider", "CategoryName": null, "LogLevel": "None", "Filter": null }, { "ProviderName": "Microsoft.Azure.WebJobs.Script.WebHost.Diagnostics.SystemLoggerProvider", "CategoryName": null, "LogLevel": null, "Filter": "<AddFilter>b__0" }, { "ProviderName": "Microsoft.Azure.WebJobs.Logging.ApplicationInsights.ApplicationInsightsLoggerProvider", "CategoryName": null, "LogLevel": "Trace", "Filter": null } ] }
Severity level: Information
6/1/2023, 7:45:04 AM
-
TRACE
ConcurrencyOptions { "DynamicConcurrencyEnabled": false, "MaximumFunctionConcurrency": 500, "CPUThreshold": 0.8, "SnapshotPersistenceEnabled": true }
Severity level: Information
6/1/2023, 7:45:04 AM
-
TRACE
FunctionResultAggregatorOptions { "BatchSize": 1000, "FlushTimeout": "00:00:30", "IsEnabled": true }
Severity level: Information
6/1/2023, 7:45:04 AM
-
TRACE
SingletonOptions { "LockPeriod": "00:00:15", "ListenerLockPeriod": "00:01:00", "LockAcquisitionTimeout": "10675199.02:48:05.4775807", "LockAcquisitionPollingInterval": "00:00:05", "ListenerLockRecoveryPollingInterval": "00:01:00" }
Severity level: Information
6/1/2023, 7:45:04 AM
-
TRACE
ScaleOptions { "ScaleMetricsMaxAge": "00:02:00", "ScaleMetricsSampleInterval": "00:00:10", "MetricsPurgeEnabled": true, "IsTargetScalingEnabled": true, "IsRuntimeScalingEnabled": false }
Severity level: Information
6/1/2023, 7:45:04 AM
-
TRACE
Starting JobHost
Severity level: Information
6/1/2023, 7:45:04 AM
-
TRACE
Starting Host (HostId=******, InstanceId=8f937b40-6d44-4919-******-ec66a372643a, Version=4.21.2.20699, ProcessId=5844, AppDomainId=1, InDebugMode=False, InDiagnosticMode=False, FunctionsExtensionVersion=~4)
Severity level: Information
6/1/2023, 7:45:04 AM
-
TRACE
Generating 1 job function(s)
Severity level: Information
6/1/2023, 7:45:04 AM
-
TRACE
Found the following functions: Host.Functions.******
Severity level: Information
6/1/2023, 7:45:04 AM
-
TRACE
Initializing function HTTP routes Mapped function route 'api/******' [get,post] to '******'
Severity level: Information
6/1/2023, 7:45:04 AM
-
TRACE
Host initialized (12ms)
Severity level: Information
6/1/2023, 7:45:04 AM
-
TRACE
Host started (12ms)
Severity level: Information
6/1/2023, 7:45:04 AM
-
TRACE
Job host started
Severity level: Information
6/1/2023, 7:45:04 AM
-
TRACE
HttpOptions { "DynamicThrottlesEnabled": true, "EnableChunkedRequestBinding": false, "MaxConcurrentRequests": 100, "MaxOutstandingRequests": 200, "RoutePrefix": "api" }
Severity level: Information
6/1/2023, 7:45:04 AM
-
TRACE
Executing 'Functions.******' (Reason='This function was programmatically called via the host APIs.', Id=1fd81ebd-e455-4a79-a9e8-5a02972a124d)
Severity level: Information
6/1/2023, 7:45:06 AM
-
TRACE
Worker process started and initialized.
Severity level: Information
6/1/2023, 7:45:06 AM
-
EXCEPTION
Exception while executing function: Functions.******
Problem Id: System.Threading.Tasks.TaskCanceledException at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess
6/1/2023, 7:45:06 AM
-
TRACE
Executed 'Functions.******' (Failed, Id=1fd81ebd-e455-4a79-a9e8-5a02972a124d, Duration=2825ms)
Severity level: Error
6/1/2023, 7:45:06 AM
-
EXCEPTION
Exception while executing function: Functions.******
Problem Id: System.Threading.Tasks.TaskCanceledException at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess
6/1/2023, 7:45:09 AM
-
TRACE
Host lock lease acquired by instance ID '3caa8b34bd20508b527345686484c415'.
Severity level: Information```

Answer 1

This is something we’ve dealt with on my enterprise team quite a bit.

What’s happening here isn’t your function failing, it’s the Azure Functions host itself timing out before your code ever runs.

If you look closely at your logs:

• The platform logs “Worker process started and initialized”

• Then immediately: System.Threading.Tasks.TaskCanceledException

• And only after that does it log the host lock lease acquisition.

The order matters. It means the request never made it to your function. The host spun up a worker for the request but didn’t get it ready in time, so it canceled the operation.

This lines up with your context: 2–3 invocations per day on a Consumption plan. FYI

• When the function app sits idle, Azure tears down the instance.

• The next HTTP call forces a cold start — loading your app, initializing the runtime, acquiring locks, etc.

• If that cold start exceeds the platform’s internal timeout, the request gets dropped with TaskCanceledException.

• And because the scale controller doesn’t guarantee sticky routing, it can route to a cold instance even if a warm one exists.

How to mitigate.

• Keep it warm: Add a lightweight timer trigger or external ping every few minutes.

• Add a warmup function (doesn't guarantee to wait until warm, just guarantee's a run of your warmup)

• Upgrade your plan: Premium or Dedicated with Always On avoids cold starts entirely.

• Pin the runtime: If this started after a platform update, pinning to a stable runtime can buy time. This has been mentioned already.

This isn’t a bug in your code, it’s a side effect of how Azure Functions scales down on idle. With a small hosting or warm-up tweak, you can eliminate these intermittent failures completely.