Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is it possible to integrate AWS SNS and AWS SQS FIFO queue

With the introduction of FIFO queues in SQS we are guaranteed that SQS messages will be delivered in order.

Now, could there be an out-of-order delivery when I publish messages through SNS? Here's what I'm thinking that could happen:

 _____                       _____                       __________
| App |-- Publish msg 1 --> | SNS | --> Queue msg 2 --> | FIFO SQS | --> Consume 2
|     |-- Publish msg 2 --> |     | --> Queue msg 1 --> |          | --> Consume 1
 -----                       -----                       ----------

Is that scenario possible?

like image 455
jlhonora Avatar asked Dec 29 '16 23:12

jlhonora


People also ask

Is SNS compatible with SQS FIFO?

You can now use Amazon Simple Notification Service (Amazon SNS) FIFO topics, in combination with Amazon Simple Queue Service (Amazon SQS) FIFO queues, to build applications that require messages to be sent and processed in a strict sequence and without duplicates.

How do SNS and SQS work together?

When you subscribe an Amazon SQS queue to an Amazon SNS topic, you can publish a message to the topic and Amazon SNS sends an Amazon SQS message to the subscribed queue. The Amazon SQS message contains the subject and message that were published to the topic along with metadata about the message in a JSON document.

Is it possible to subscribe and SQS queue to an SNS topic?

You can subscribe one or more Amazon SQS queues to an Amazon Simple Notification Service (Amazon SNS) topic. When you publish a message to a topic, Amazon SNS sends the message to each of the subscribed queues.

Can SNS read from SQS?

SQS cannot publish messages to SNS. SQS can only store the messages. You have to pull the message using SQS Api's.


3 Answers

This scenario is currently impossible because SNS is not compatible with FIFO SQS in AWS. It is only supported for standard queues. This is clearly mentioned in the AWS docs in this link, under special notes.

EDIT

It's also listed in the SNS FAQs, under Are Amazon SQS FIFO queues compatible with Amazon Simple Notification Service (SNS)?

The answer's still the same as of July 26, 2020: Amazon SNS does not currently support forwarding messages to Amazon SQS FIFO queues. You can use SNS to forward messages to standard queues.

like image 191
Keet Sugathadasa Avatar answered Oct 09 '22 06:10

Keet Sugathadasa


SNS FIFO just launched. SNS can now integrate with SQS FIFO https://aws.amazon.com/blogs/aws/introducing-amazon-sns-fifo-first-in-first-out-pub-sub-messaging/

like image 22
Marc Yu Yang Avatar answered Oct 09 '22 06:10

Marc Yu Yang


It is possible with the latest SNS support for FIFO,

As per the excerpt from amazon doc

FIFO topics support fanning out messages to multiple subscriptions with high durability, filtering, encryption, and privacy, while FIFO topics provide the added benefit of ordering and deduplication of message

I have listed some important links for you to refer

  1. Amazon SNS introduces First-in-First-out (FIFO) topics with strict ordering and deduplication of messages
  2. Introducing Amazon SNS FIFO – First-In-First-Out Pub/Sub Messaging
  3. Building event-driven architectures with Amazon SNS FIFO

As your SNS topic protocol endpoint is Amazon SQS which perfectly works fine.

NOTE for other protocol endpoints Ref

  • Currently, the endpoint protocol must be Amazon SQS, with an Amazon SQS FIFO queue's Amazon Resource Name (ARN) as the endpoint.
  • SNS FIFO topics can't deliver messages to customer managed endpoints, such as email addresses, mobile apps, phone numbers for text messaging (SMS), or HTTP(S) endpoints. These endpoint types aren't guaranteed to preserve strict message ordering. Attempts to subscribe customer managed endpoints to SNS FIFO topics result in errors.
like image 34
Prasanth Rajendran Avatar answered Oct 09 '22 06:10

Prasanth Rajendran