'Queueing' tutorials and documentation?

Question

I'm looking for articles and references that give an overview of 'queueing' (I'm probably not even using the right term here). I'm hoping for an introductory styled guide through a world of Redis, RabbitMQ, Celery, Kombu, and whatever other components exist that I haven't read about yet, and how they fit together.

My problem is I need to queue up background tasks for issued by my Django website, and every blog and article I read recommend different solutions.

Answer 1

Lots of options available to you, and your choice will likely come down to personal preference and what dependencies you feel comfortable installing.

I'll give a vote for Redis. I evaluated RabbitMQ, ActiveMQ, HornetQ, and Redis and found Redis to offer the best mix of ease of installation, simplicity, and performance.

It's technically not a message queue, but the push/pop primitives for the list types provide atomic queue-like operations, so it can effectively be used as a queue. It has worked well for us.

One python specific project on top of Redis you might look at:

http://richardhenry.github.com/hotqueue/tutorial.html

Very simple. But again, all the other options, like Celery, are viable too.