Does Pub/Sub immediately resend a message in case of a non-success response?

Question

I have a Cloud Function subscribed to a Pub/Sub topic.

According to the docs, in case the Cloud Function returns a non-success response, it will resend the message:

The endpoint acknowledges the message by returning an HTTP success status code. A non-success response indicates that the message should be resent.

My question is: In case of a non-success response, the message will be immediately resent or Pub/Sub will respect the confirmation deadline to resend the message?

Update:

I think this question makes no sense, and I'll explain why.

Since you have no access to the Response object in Pub/Sub functions, there's no way to send a response code directly (I'm so not sure about that, so please correct me if I'm wrong).

Because of that, a non-success response can only be caused by:

1. Timeout, which is based on the confirmation deadline, so in this case the confirmation deadline is obviously going to be respect.

2. An error in the Cloud Function, and as Andrei Cusnir said, by default Pub/Sub won't even retry, so caring about the confirmation deadline makes no sense.

Answer 1

While investigating the Google Cloud Functions behavior, I found out that if the Cloud Function fails and responds with an error message it will not retry the operation. I have created a Cloud Function that would be triggered by the Pub/Sub event and then respond with an error message. The message was logged in the Stackdriver logging page and then the error was logged as well but the function never resent the message so it was never re-triggered. After that, I also tried responding with error code before actually reading the message of Pub/Sub and it still failed in the logs but never resent the message to actually retry its operation again.

This is expected behavior for the Google Cloud Function to protect themselves from running in an infinitive loop. If the Cloud Function based on the Pub/Sub topic event trigger, failed, it most probably will fail again during retrying. So if it keeps calling itself all the time it resends the message to the topic, it will be stuck in a loop generating billing costs as well.

To use the retry feature of a Google Cloud Function, you will have to enable it when deploying, using a flag. For more information, you can see at Retrying Background Functions documentation. Based on what you will set up in the configuration for the Google Cloud Function retrying strategy, it will trigger a retry.

However, the retry will be immediate and it will not respect the confirmation deadline.

UPDATE

When creating a Google Cloud Function with Pub/Sub event trigger, a subscription is created for the chosen topic, as you can see this in the topic's details page (Google Cloud Platform > Pub/Sub > Topics > [TOPIC_NAME]). The subscription is responsible for reading the message published in the topic and sending the ack back to the topic. A topic can have multiple subscriptions and it is sending each received message at least once to each subscription. As it is stated in Subscriber overview documentation.

In the same link, it is stated that A message is considered outstanding once it has been sent out for delivery and before a subscriber acknowledges it.. It is also stated that The subscriber has a configurable, limited amount of time -- known as the ackDeadline -- to acknowledge the outstanding message.. To elaborate further on how this system works, imagine having a TopicA then SubscriberA and SubscriberB subscribed to that topic. Now, a message is being sent to TopicA, so TopicA forwards the message to Subscriber-A and Subscriber-B and starts waiting for acknowledgments from both subscribers. Each subscriber has a different ack response time which is ackDeadline. If SubscirberA responds with ack, TopicA will keep waiting for Subscriber-B. If Subscriber-B will not send an ack within the time specified, then Topic-A will resend the message only to Subscriber-B.

Now, you have mentioned the The endpoint acknowledges the message by returning an HTTP success status code. A non-success response indicates that the message should be resent. part of the documentation. This part is talking about the subscriber's response and not the Google Cloud Function's response. Since the Google Cloud Function was called due to the message being published in the Topic, from that time the subscriber associated with that Cloud Function will send the ack message back to the topic. After that either the Google Cloud Function fails to process the message or not, it will not affect in any way the Pub/Sub topic as it only cares for the message being already delivered.

To elaborate further on the two points in your update section of the question:

1. If Google Cloud Function will be executed, this means that the ack message was already been sent back to the topic. Imagine having a subscriber with 10 seconds set as ack response time and a Cloud Function that needs 5 minutes to time out. Obviously, when the 10 seconds of the subscriber has passed already, it will not wait 4 minutes and 50 seconds more for the Cloud Function to timeout. Therefore it means that the ack was already been sent by the subscriber as soon as the Google Cloud Function was called.
2. The error that I have mentioned in my first response is the Google Cloud Function's error. So if your Cloud Function failed to process the received message, you will have to implement a retry method that will retry a specific amount of times and then will stop to avoid looping. However, the topic has already been informed that the message was delivered to the subscriber and therefore the acknowledgment was already been sent the time it started the Cloud Function execution.

Answer 2

Cloud Functions relies on Cloud Pub/Sub Push deliveries to trigger execution. The rate at which the system delivers concurrent messages continues to double until there is a delivery failure or the system reaches a quota or resource limit. For each delivery failure, the number of concurrent requests to the endpoint halves.

Cloud Pub/Sub, however, will not wait until the ack expiration deadline to resend the message if it was explicitly nacked by the non-ok status. Instead, it will temporarily back-off pushing messages. You can expect this delay to be in the order of a few seconds after a number of delivery failures.

You can read more about quotas and delivery rate here.