How to debug issues with Amazon SQS subscription to SNS

Question

I want to build a pub/sub messaging system into my services that are hosted on Amazon Web Services, and creating SQS queues that subscribe to SNS topics seems like the obvious direction to take, but I can't get it working at all.

So far my code looks for the topics and the queues at startup and creates anything that's missing. This seems to work, I can see the SNS topic and the SQS queues in the AWS management console, and I can see that the queue is subscribed to the topic, but when I publish messages to the topic nothing ends up in the queue.

Even if I manually publish a message using the 'Publish' button in the management console the queue is still empty.

I changed the permissions on both the topic and the queue to 'everyone can do everything' just to eliminate this possibility. My code receives a message ID in response to the publish and there are no errors, every API call returns a 200 (OK) status.

Where can I go from here to figure out why it's not working?

Answer 1

The SNS --> SQS link has a few gotchas:

• "Amazon SNS isn't currently compatible with FIFO queues." per the note on their Subscribing an Amazon SQS Queue to an Amazon SNS Topic Tutorial
• You have to fiddle with the IAM permissions (see the page on Sending Amazon SNS Messages to Amazon SQS Queues)
• You can't send messages to encrypted queues (see their Server-Side Encryption page)

It would definitely have been easier to figure this out if all this info were consolidated into a single page. The killer for me was #3 - perhaps one of these will be the solution to your issue.

Answer 2

A couple of options -

1. Enable CloudTrail and monitor the logs
2. View the CloudWatch logs to identify any permissions issues
3. Open a ticket with AWS support.

Ideally, you wouldn't be creating the resources in your application but instead decouple those into CloudFormation or at a minimum CLI scripts. If you require the ability to dynamically create these resources, using the AWS IoT Message Broker may be a better option since it supports ephemeral messaging resources - http://docs.aws.amazon.com/iot/latest/developerguide/iot-message-broker.html