Menu
Im trying to create infinite job queue using redis and ruby eventmachine. To achieve that Im using redis BLPOP command with 0 timeout. After successful BLPOP I run it again.
Am I on the right way or there is a better way to create job queue with redis?
262
Using Redis with Redis Queue allows you to enter those complex tasks into a queue, so the Redis worker executes these tasks outside of your application's HTTP server. In this article, we will build an app that enqueues jobs with Redis queue, performs a function on those jobs and returns the result of the function.
BLPOP is a blocking list pop primitive. It is the blocking version of LPOP because it blocks the connection when there are no elements to pop from any of the given lists. An element is popped from the head of the first list that is non-empty, with the given keys being checked in the order that they are given.
If you use BLPOP alone to remove a message from the queue, and your message consumer fails to process it, the message will have to be re-queued, lest it disappear forever along with the failed consumer.
For more durable message processing, a list of messages being processed must be maintained so they can be re-queued in the event of failure.
[B]RPOPLPUSH is perfect for this scenario; it can atomically pop a message from the message queue and pushes it onto a processing queue so that the application can respond in the case of a failure on the consumer's end.
http://redis.io/commands/rpoplpush
Actual re-queueing is left to the application, but this redis command provides the foundations to do so.
There are also some drop-in-place implementations of queues using redis floating around the web, such as RestMQ [ http://www.restmq.com/ ]
171
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
