How to shuffle jobs in a Resque queue?

Question

I have a queue named check_integrity and lots of jobs in it. When i run a worker for it it takes jobs first in first out order. Is it possible to shuffle the jobs in that particular queue? I need the worker to take jobs randomly. Please help.

Thanks.

Answer 1

One way to go about this is by popping entries out of the queue, batching them up, shuffling the batch and then re-insert them:

key = "resque:queue:bulk"
total = Redis.current.llen(key)
batch_size = 5_000 # any value that is good enough for you

batch = []
total.times do |i|
  entry = Redis.current.lpop(key)
  batch << entry
  if batch.size == batch_size || i.succ == total
    puts "re-inserting batch..."
    Redis.current.rpush key, batch.shuffle
    batch = []
  end
end

This is really useful when you mistakenly enqueue several jobs that end up racing for shared resources, locks and so on.

Answer 2

Look at this plugin for the Resque. I guess this is exactly what you need.