Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do I deal with 'poison pill' messages when using Google Pub/Sub?

When using Amazon's SQS for example, I can define a dead letter queue (DLQ) where any message that has failed to be deleted (ack'd) after X retries will be routed to for separate processing.. but it Google Cloud Platform I don't see any mention of this

like image 481
1977 Avatar asked Feb 24 '17 10:02

1977


2 Answers

Google Cloud Pub/Sub does not currently have any automatic dead letter queues. If you are worried about "poison pill" messages, you will have to support this in some capacity yourself by persistently keeping a map from the message ID to the number of times the message has been delivered. You would update this map in your subscriber before reading the data in the message. Once acknowledged, you remove from the map. If the count exceeds some threshold, you could publish it to a separate Cloud Pub/Sub topic that you use to keep track of such messages and then ack the message.

like image 90
Kamal Aboul-Hosn Avatar answered Nov 20 '22 01:11

Kamal Aboul-Hosn


Cloud Pub/Sub now supports Dead Letter Queues that can be used to handle poison pill messages.

like image 3
Mahesh Gattani Avatar answered Nov 20 '22 00:11

Mahesh Gattani