Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Can I use Amazon SQS as a delay queue before sending to SNS?

My system run on an Amazon autoscaling group and one feature allows user to user messaging and I have the following use case to resolve.

  1. A new message is sent in my application between users.
  2. A message to notify the the user by e-mail is dropped into a queue with a 60 second delay. This delay allows time for a realtime chat client (faye/angularjs) to see the message and mark it as viewed.
  3. After the delay the message is picked up, the "read" status is checked and if it has not been read by the client an e-mail is dispatched.

Originally I was going to use a cronjob on each application server poll the message queue however it occurs to me it would be more efficient to use SNS to call some kind of e-mail sending endpoint (perhaps in Lambda).

I can't see any way to have SNS poll SQS however, can anybody suggest how this could be done? Essentially I want SNS with a delay so that I don't spam somebody in a "live" chat with e-mail alerts.

Thanks

like image 688
JimBlizz Avatar asked Sep 02 '15 07:09

JimBlizz


People also ask

Under what circumstances would you use an SQS delay queue?

Delay queues let you postpone the delivery of new messages to consumers for a number of seconds, for example, when your consumer application needs additional time to process messages. If you create a delay queue, any messages that you send to the queue remain invisible to consumers for the duration of the delay period.

Can SQS be used without SNS?

If you want unknown number and type of subscribers to receive messages, you need SNS. You don't have to couple SNS and SQS always. You can have SNS send messages to email, SMS or HTTP end point apart from SQS. There are advantages to coupling SNS with SQS.

Can SQS push to SNS?

SQS cannot publish messages to SNS. SQS can only store the messages. You have to pull the message using SQS Api's. Hope this helps you!

How do SQS and SNS work together?

When you subscribe an Amazon SQS queue to an Amazon SNS topic, you can publish a message to the topic and Amazon SNS sends an Amazon SQS message to the subscribed queue. The Amazon SQS message contains the subject and message that were published to the topic along with metadata about the message in a JSON document.


1 Answers

Unfortunately this is not yet available out of the box. The missing part is the generation of Amazon SNS notifications on message arrival/visibility by an Amazon SQS queue, be it via push (similar to Amazon S3 notifications, or via poll similar to Amazon Kinesis subscriptions (see The Pull/Push Event Models for more on the difference), which would both allow to directly connect an AWS Lambda function to the resp. SQS delay queue events, see e.g.:

  • Lambda with SQS

That being said, you can work around this limitations in a few ways, for example:

  • trigger your Lambda function on schedule (e.g. once per minute), and poll your SQS delay queue from there
    • scheduled Lambda functions are an eagerly awaited missing Lambda feature in turn, but it is more easily worked around, be it either by a cron job of yours, or Eric Hammond's Unreliable Town Clock (UTC) for example

The AWS Lambda team has delivered many/most similar feature requests over recent month' btw., so I would expect them to offer both SQS event handling and scheduled Lambda functions over the course of the year still.

like image 83
Steffen Opel Avatar answered Sep 29 '22 08:09

Steffen Opel