Is there a FIFO message queuing service offering the high availability of Amazon SQS?

Question

Would have loved to use Amazon SQS if it provided some semblance of FIFO access, but the sequence seems to completely random.

Is there something that would provide me FIFO queuing as-a-cloud-service with the high availability of SQS?

If that is asking for too much - what would be the easiest way of putting together something with the above requirements in EC2? Or maybe in other words, what's the easiest highly available queuing solution that works in EC2?

Thanks for your insights!

Answer 1

Update 2016-11-19

Amazon SQS has just gained FIFO Queues with Exactly-Once Processing & Deduplication:

Today we are making SQS even more powerful and flexible with support for FIFO (first-in, first-out) queues. We are rolling out this new type of queue in two regions now, and plan to make it available in many others in early 2017.

These queues are designed to guarantee that messages are processed exactly once, in the order that they are sent, and without duplicates. [...]

[emphasis mine]

As emphasized, these new FIFO SQS queues will cover the use case at hand, but are not yet available in all SQS regions [initially only in US East (Ohio) and US West (Oregon)]. Also, the SQS FAQ for FIFO queues outlines notable differences between standard and FIFO queues that should be considered upfront, for example a throughput limit of 300 transactions per second.

---

Initial Answer

Would have loved to use Amazon SQS if it provided some semblance of FIFO access, but the sequence seems to completely random.

While I haven't experienced completely random message ordering yet (this likely depends on the use case and especially the message volume though), there is no FIFO guarantee indeed, see the respective FAQ Does Amazon SQS provide first-in-first-out (FIFO) access to messages?:

No, Amazon SQS does not guarantee FIFO access to messages in Amazon SQS queues, mainly because of the distributed nature of the Amazon SQS. If you require specific message ordering, you should design your application to handle it.

Given you expressed interest in hosted RabbitMQ as well as StormMQ, I might as well point you to other commercial offerings:

CloudAMQP

CloudAMQP is RabbitMQ as a Service, thus exactly what you have been asking for, given RabbitMQ supports the desired true FIFO message ordering (see Amazon SQS vs. RabbitMQ for a nice comparison).

According to their Plans & Prices, it is apparently only offered as an addon to first class platform providers though, hence you'll have to look into these in turn:

• AppHarbor CloudAMQP Add-on
• Heroku CloudAMQP Add-on
• cloudControl CloudAMQP Add-on

IronMQ

IronMQ offers developers ready-to-use messaging with highly reliable delivery and cloud-optimized performance. It is complying to nowadays expectations on a Software as a service (SaaS) product, not the least regarding an easy to understand and especially published pricing model As rightfully critizised by Sleavely, Iron.io has seemingly dropped its former exemplary pricing model - see Alex Payne's How Not To Sell Software in 2012 for a nice rant and advise regarding this.

I've only tested it shortly myself so far, but have been pretty pleased with the offered features and language integrations (see Client Libraries and Beanstalkd Support) - given their competitive price tag and especially the free tier makes it a good candidate for exploring a message queuing as a service solution, not the least in combination with their second product IronWorker (An easy-to-use and massively scalable task queue [...]), which provides functionality not even available from AWS as of today.

FIFO message ordering

Unfortunately I haven't been able to figure out whether true FIFO is supported by IronMQ directly, consequently I actually doubt it - accordingly, you'd need to file a support request to verify this.

Evan Shaw from Iron.io confirmed that IronMQ provides FIFO message ordering in fact (thanks much).