Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Migrate from AMQP to Amazon SNS/SQS - need to understand concepts

I am well experienced with the RabbitMQ and AMQP protocol, and have built a system with patterns for Commands, Requests and Events.

Now I am going to build a system running on AWS Lambda and therefore use SNS, SQS etc. I want to understand the "mapping" between these things.

What are the equivalent to an exchange in AMQP? What are the equivalent to a routing key?

How to set up queue bindings for fanout, direct and topic exchanges (or similar) in SNS and SQS?

How did other people handle this? To me it looks like RabbitMQ is a tool built to fit the usual needs of a message bus, where AWS provides blocks and you have to setup/build the functionality yourself. Am I right?

like image 328
Esben von Buchwald Avatar asked Oct 22 '17 23:10

Esben von Buchwald


People also ask

Do we need SQS for SNS?

If you want unknown number and type of subscribers to receive messages, you need SNS. You don't have to couple SNS and SQS always. You can have SNS send messages to email, SMS or HTTP end point apart from SQS. There are advantages to coupling SNS with SQS.

What protocol does AWS SQS use?

Amazon SQS supports the HTTP over SSL (HTTPS) and Transport Layer Security (TLS) protocols. Most clients can automatically negotiate to use newer versions of TLS without any code or configuration change. Amazon SQS supports versions 1.0, 1.1, and 1.2 of the Transport Layer Security (TLS) protocol in all regions.

How does SNS and SQS work together?

When you subscribe an Amazon SQS queue to an Amazon SNS topic, you can publish a message to the topic and Amazon SNS sends an Amazon SQS message to the subscribed queue. The Amazon SQS message contains the subject and message that were published to the topic along with metadata about the message in a JSON document.

What type of queuing system is Amazon SQS?

Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications.


2 Answers

What are the equivalent to an exchange in AMQP?

The closest concept might be SNS, as you can configure a SNS topic to publish to n SQS queues. Then when you write to that topic, each subscribed queue gets a message. You can also write messages directly to SQS queues if you like.

What are the equivalent to a routing key?

There's no real equivalent for this. The SNS-to-SQS bindings don't allow for any additional filtering/control beyond topic-to-queue bindings. You could approximate routing by having multiple SNS topics i.e. each topic is a "routing key".

How to set up queue bindings for fanout, direct and topic exchanges (or similar) in SNS and SQS?

  • Fanout: write to SNS topic and every subscribed queue will receive the same message.
  • Direct: write directly to SQS queue(s), or SNS topic that has only those queues subscribed.
  • Topic: create SNS topic and subscribe queues accordingly.

How did other people handle this?

I used RabbitMQ before AWS messaging, so I went through the same learning curve. AWS doesn't provide as many exchange/routing bells & whistles, but in my experience you can get close enough with some combination of SNS topics and SQS queues.

like image 168
Taylor Wood Avatar answered Nov 06 '22 03:11

Taylor Wood


It looks like the AWS IoT service with its MQTT provides what I need in order to do routing rules similar to the ones RabbitMQ provides!

like image 36
Esben von Buchwald Avatar answered Nov 06 '22 01:11

Esben von Buchwald