Menu
What is the difference between setting a Dead Letter Queue on an SNS Topic or on a Lambda function?
I was wondering, because if you set the DLQ on the SNS subscription, then that subscription message will fail over to DLQ when the Lambda (the subscriber) fails, correct? So in that scenario would setting the DLQ in these two places have the same effect?
I have set a DLQ on a SNS Topic Subscription, and it didn't "automagically" appear as the DLQ on the Lambda screen settings, so I assume there may be some difference?
SNS dead letter queue ref: https://docs.aws.amazon.com/sns/latest/dg/sns-dead-letter-queues.html
In general, message delivery fails when Amazon SNS can't access a subscribed endpoint due to a client-side or server-side error.
Lambda dead letter queue ref: https://aws.amazon.com/about-aws/whats-new/2016/12/aws-lambda-supports-dead-letter-queues/
AWS Lambda will write the event object invoking the Lambda function to this [DLQ] endpoint after the standard retry policy (2 additional retries on failure) is exhausted.
Lambda:
SNS subscription:
469
A dead-letter queue is attached to an Amazon SNS subscription (rather than a topic) because message deliveries happen at the subscription level. This lets you identify the original target endpoint for each message more easily.
Having SQS in between SNS and Lambda allows reprocessing older unprocessed events in case Lambda fails to process. SQS allows to put a delay, so that message gets processed after some time, it may be useful in the scenario where data takes time to be available.
AWS SNS is a publisher subscriber network, where subscribers can subscribe to topics and will receive messages whenever a publisher publishes to that topic. AWS SQS is a queue service, which stores messages in a queue.
DLQ is a dead letter queue which helps to prevent message loss in the case of failure during the lambda function invocations. SNS Topic is use to publish the messages. It asynchronously deliver the message every time when you publish a message on SNS.
The SNS DLQ is specific for SNS, the benefit here is that it also accounts for client errors such as the Lambda service being down. This will ensure if the Lambda service is down the messages can be replayed to it later, whereas if the DLQ is attached to the Lambda this would only account for a replay if the service is running.
However as I mentioned the SNS DLQ is only for notifications that are from SNS, whereas Lambda can support the DLQ from any incoming events. This means if you have multiple SNS topics, or an SNS topic and some SQS queues you only need to apply it to the Lambda itself.
Both services use SQS for their DLQs so the ingestion/retrieval from both would be identical. If you have a DLQ on both services then yes you might end up with 2 copies of the event/notification, however it is unlikely that you will get both as in theory would the Lambda endpoint acknowledges it SNS would treat as sent, this would then be Lambdas responsibility to add to the DLQ if it fails.
106
SNS DLQ and Lambda DLQ protect your workload from different failure modes. The former is for failed message delivery from a topic, whereas the latter is for failed function execution. You should use both simultaneously. Here more info: https://aws.amazon.com/blogs/compute/designing-durable-serverless-apps-with-dlqs-for-amazon-sns-amazon-sqs-aws-lambda/
44
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
