Message Queues in Ruby on Rails

Question

What message queues are people using for their Rails apps and what was the driving force behind the decision to choose it. Does the latest Twitter publicity over their in house queue Starling falling down affect any existing design decisions.

I am working on an app that will need a message queue to process some background tasks, I haven't done much of this, and most of the stuff I have seen in the past has been about Starling and Workling, and to be honest the application is not very big and this solution would probably suffice, but I'd love to get experience integrating the best solution possible as I'm sure I will integrate one into a bigger app at some point.

What message queues would you suggest for a Rails app???

EDIT: Thanks for the suggestions, I'm going to look at a few of them this weekend.

EDIT Again: I've had a look around and a little overwhelmed for choice. I am however going to go about integrating RabbitMQ with Workling into the app I am building, then if I ever need some knowledge about a fast queue then I will have this and know whether or not it fits my needs.

EDIT: Finding more and more that DJ suits me just fine, if I ever "outgrow" it on a site I'd say that Resque is where I would head.

EDIT: (Dec 2014) So it's been a long time since I asked this, but I see it still gets some views or some votes, so I figured I'd update it on my approach now when it comes to my choice of background workers.

In my opinion currently the best way to run background jobs in Ruby is using Sidekiq. A lot of people have really lauded Sidekiq for it's threaded workers rather than process per worker which can use significantly less memory than the likes of Resque, which I was using before Sidekiq. This is good but for me this wasn't the killer feature. By using Sidetiq with Sidekiq, the scheduling of jobs is so trivial that I switched over and have never looked back from it, by far the easiest scheduling of jobs that I have used and has made Sidekiq a breeze to use.

Answer 1

As an update -- GitHub have moved to Resque on Redis instead of Delayed job. However they still recommend delayed_job for smaller setups:

https://github.com/resque/resque

Answer 2

Chris Wanstrath from github was at the SF Ruby meetup recently, talking about their queue. They tried Starling, beanstalk, and some other variants before settling on Shopify's delayed_job. They are pretty aggressive with their use of backgrounding.

Here's a blog post from last year that talks about their move to DJ.

Where I am now we rolled our own several years ago, but I'm taking some ideas from DJ to improve the handling.