Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Amazon SQS Priority Queue

Is it possible to create a priority queue using the Amazon simple queuing service?

Initially I couldn't find anything on the subject and this is why I create two queues. A general queue and a priority queue. I am enqueuing messages to this queue according the rule I've defined but confusion arises while dequeuing messages.

How do I do a long poll on the queues such that my combination of queues behave like a single priority queue?

like image 451
Karan Verma Avatar asked Mar 19 '14 01:03

Karan Verma


People also ask

Does SQS have priority queue?

Priority: Use separate queues to provide prioritization of work. Scalability: Because message queues decouple your processes, it's easy to scale up the send or receive rate of messages — simply add another process. Resiliency: When part of your system fails, it doesn't need to take the entire system down.

How do you implement priority queue in SQS?

Implementing the priority queue pattern is easy in a polling environment: all we have to do is poll the priority queue first. If it has work available, process it. If the priority queue is empty, poll the main queue. AWS Lambda and many other serverless technologies are event-driven.

Can we set priority to SQS message?

Client can specify a priority of 0-255 in the message, and the queue will prioritize a higher priority message gets to the customer first. Save this answer. Show activity on this post. By "when a msg fails", if you meant "processing failure" then you could look into Dead Letter Queue (DLQ) feature that comes with SQS.

What are the two types of AWS SQS queues?

Amazon SQS supports two types of queues – standard queues and FIFO queues. Use the information from the following table to chose the right queue for your situation. To learn more about Amazon SQS queues, see Amazon SQS Standard queues and Amazon SQS FIFO (First-In-First-Out) queues.


2 Answers

I think you're on the right track by creating two queues - A normal queue and a priority queue.

You don't necessarily need a long poll in this case. Since messages in the priority queue take precedence over messages in the normal queue, you could follow an approach like this:

  1. Poll priority queue till there are no more messages.
  2. Poll normal queue and repeat step 1 after every message in the normal queue.
like image 154
Nikhil Avatar answered Oct 14 '22 01:10

Nikhil


You could use Shoryuken and set a higher weight to configure the priority:

queues:
  - [high_priority_queue, 5]
  - [low_priority_queue, 1]
like image 28
Pablo Cantero Avatar answered Oct 14 '22 01:10

Pablo Cantero