Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Most effective way to poll an Amazon SQS queue using Node

My question is short, but I think is interesting:

I've a queue from Amazon SQS service, and I'm polling the queue every second. When there's a message I process the message and after processing, go back to polling the queue.

Is there a better way for this?, some sort of trigger? or which approach will be the best in your opinion, and why.

Thanks!

like image 725
Laerion Avatar asked Jul 08 '15 21:07

Laerion


People also ask

Which is generally preferred SQS long polling or SQS short polling?

From SQS FAQs: In almost all cases, Amazon SQS long polling is preferable to short polling. Long-polling requests let your queue consumers receive messages as soon as they arrive in your queue while reducing the number of empty ReceiveMessageResponse instances returned.

What must be done for long polling to be enabled in SQS?

You can enable long polling when receiving a message by setting the wait time in seconds on the ReceiveMessageRequest that you supply to the AmazonSQS class' receiveMessage method.

What is poll in SQS?

Amazon SQS provides short polling and long polling to receive messages from a queue. By default, queues use short polling. With short polling, the ReceiveMessage request queries only a subset of the servers (based on a weighted random distribution) to find messages that are available to include in the response.

Which Amazon SQS queue type offers maximum throughput best effort ordering and at least once delivery?

SQS offers two types of message queues. Standard queues offer maximum throughput, best-effort ordering, and at-least-once delivery. SQS FIFO queues are designed to guarantee that messages are processed exactly once, in the exact order that they are sent.


2 Answers

A useful and easily to use library for consuming messages from SQS is sqs-consumer

const Consumer = require('sqs-consumer');

const app = Consumer.create({
  queueUrl: 'https://sqs.eu-west-1.amazonaws.com/account-id/queue-name',
  handleMessage: (message, done) => {
    console.log('Processing message: ', message);
    done();
  }
});

app.on('error', (err) => {
  console.log(err.message);
});

app.start();

It's well documented if you need more information. You can find the docs at: https://github.com/bbc/sqs-consumer

like image 113
nickool Avatar answered Oct 21 '22 09:10

nickool


yes there is: http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-long-polling.html

you can configure the SQS queues to have a "receive message wait time" and do long polling.

so you can set it to say 10 seconds, and the call will come back only if you have a message or after the 10 sec timeout expires. you can continuously poll the queue in this scenario.

like image 36
Mircea Avatar answered Oct 21 '22 09:10

Mircea