How to check running status and stop Durable function

Question

I want to process millions of records on-demand, which takes approximate 2-3 hours to process. I want to go serverless that is why tried durable function (first-time). I want to check, how long I can run durable function so I created 3 functions

1. Http function to kick start Orchestrator function
2. Orchestrator function
3. Activity function

My DurableFunction is running and emitting logs in Application Insights from last 5 days and based on my code it would take 15 more days to complete.

I want to know that how to stop Orchestrator function manually?

I can see thousands of entry in ApplicationInsights requests table for single execution, Is there any way to check how many DurableFunction running in backend? and how much time taken by single execution?

I can see some information regarding orchestrator function in "DurableFunctionHubInstance" table but as MS recommended not rely on table.

Answer 1

Since Durable Functions does a lot of checkpointing and replays the orchestration, normal logging might not always be very insightful.

Getting the status

There are several ways to query for the status of orchestrations. One of them is through the Azure Functions Core tools as George Chen mentioned.

Another way to query the status is by using the HTTP API of Durable Functions directly:

GET <rooturl>/runtime/webhooks/durableTask/instances?
    taskHub={taskHub}
    &connection={connectionName}
    &code={systemKey}
    &createdTimeFrom={timestamp}
    &createdTimeTo={timestamp}
    &runtimeStatus={runtimeStatus1,runtimeStatus2,...}
    &showInput=[true|false]
    &top={integer}

More info in the docs.

The HTTP API also has methods to purge orchestrations. Either a single one by ID or multiple by datetime/status.

DELETE <rooturl>/runtime/webhooks/durabletask/instances/{instanceId}
    ?taskHub={taskHub}
    &connection={connection}
    &code={systemKey}

Finally you can also manage your instances using the DurableOrchestrationClient API in C#. Here's a sample on GitHub: HttpGetStatusForMany.cs

I have written & vlogged about using the DurableOrchestrationClient API in case you want to know more about how to use this in C#.

Custom status

Small addition: it's possible to add a custom status object to the orchestration so you can add enriched information about the progress of the orchestration.

Getting the duration

When you query the status of an orchestration instance you get back a DurableOrchestrationStatus object. This contains two properties:

• CreatedTime
• LastUpdatedTime

I'm guessing you can subtract those and get a reasonable indication of the time it has taken.

Answer 2

You could manage the Durable Functions orchestration instances with Azure Functions Core Tools.

Terminate instances:

func durable terminate --id 0ab8c55a66644d68a3a8b220b12d209c --reason "It was time to be done."

Query instances with filters: you could add the parameter(runtime-status) to filter the running instances.

func durable get-instances --created-after 2018-03-10T13:57:31Z --created-before  2018-03-10T23:59Z --top 15

As for the time functions took, looks like it doesn't support. The similar parameter is the get-history.