Menu
I am trying to use x-ray to trace requests which use an SNS-SQS fanout pattern.
The request comes from API GW, lambda proxy integration, published to SNS and delivered to a subscribed SQS which has a lambda trigger which receives the messages for further processing.
However the trace stops at SNS.
935
The Fanout scenario is when a message published to an SNS topic is replicated and pushed to multiple endpoints, such as Kinesis Data Firehose delivery streams, Amazon SQS queues, HTTP(S) endpoints, and Lambda functions. This allows for parallel asynchronous processing.
The typical way to fanout messages to multiple sqs queues is to use SNS. The s3 event notifications would goto SNS instead of SQS and the SNS would be responsible for fanning those messages out to as many queues as you want.
SQS is mainly used to decouple applications or integrate applications. SNS is used to broadcast messages and it's up to the receivers how they interpret and process those messages.
SQS offers two types of message queues. Standard queues offer maximum throughput, best-effort ordering, and at-least-once delivery. SQS FIFO queues are designed to guarantee that messages are processed exactly once, in the exact order that they are sent.
Unfortunately we do not support this architecture today. The issue is that the trace information from the starting request (APIG in this case) is lost once the SNS message is invoked. There currently isn't a workaround for this behavior. We are working with SNS and SQS to provide a better user experience and support for these cases. Please stay tuned for more.
177
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
