Cloud Tasks + Cloud Functions - duplicate executions

Question

I'm using Google Cloud Tasks and Cloud Functions together to execute a lot of tasks in parallel.

• The Cloud Function is deployed with an HTTP trigger.
• I'm creating multiple HTTP target tasks, in one queue, to call the same Cloud Function for several different inputs.
• The task done by the Cloud Function involves calling HTTP APIs that can occasionally fail and need retrying. For that reason, the task queue has maxAttempts = 10.
• This process runs once per day, and it generates about 11 thousand tasks throughout 30 or so minutes.

It seems to work well most of the time, but I've seen an unexpected behavior happen a few times: sometimes, Cloud Tasks thinks that the function failed and retries it, even though the function hadn't finished running yet, and eventually succeeded. When that happens, I see this in the queue's logs (the queue was created with --log-sampling-ratio=1.0):

• Cloud Tasks creates a task.
• Dispatch 1 is attempted.
• Cloud Function execution 1 starts.
• Cloud Function execution 1 crashes. The queue's log shows that attempt 1 failed with an "UNAVAILABLE" status, and dispatch 2 is attempted, as expected.
• Cloud Function execution 2 starts.
• Even though execution 2 hasn't finished yet, the queue's log shows that dispatch attempt 2 failed with an "UNAVAILABLE" status, then dispatch 3 is attempted.
• Cloud Function execution 3 starts.
• Cloud Function execution 3 finishes with status 200.
• Cloud Function execution 2 finishes with status 200.

So the Cloud Function ended up executing with success 2 times.

I know that Cloud Tasks can't guarantee strict only-once execution at all times, as stated in the documentation. However, I'd still like to confirm if this particular occurrence is expected and whether there is any way to prevent it.

Answer 1

Define in your task:

tasksClient.createTask({ parent: queuePath, task, resource: { retryConfig: { maxAttempts: 1 } } })

I found it here: doc