Programmatic access to the Resque failed-job queue

Question

How can I write code to go through the Resque failure queue and selectively delete jobs? Right now I've got a handful of important failures there, interspersed between thousands of failures from a runaway job that ran repeatedly. I want to delete the ones generated by the runaway job. The only API I'm familiar with is for enqueuing jobs. (I'll continue RTFMing, but I'm in a bit of a hurry.)

Answer 1

I neded up doing it like this:

# loop over all failure indices, instantiating as needed  
(Resque::Failure.count-1).downto(0).each do |error_index_number|     

  failure = Resque::Failure.all(error_index_number)

  # here :failure is the hash that has all the data about the failed job, perform any check you need here      

  if failure["error"][/regex_identifying_runaway_job/].present?        
    Resque::Failure.remove(error_index_number)
    # or
    # Resque::Failure.requeue(error_index_number)
  end

As @Winfield mentioned, having a look at Resque's failure backend is useful.

Answer 2

You can manually modify the Failure queue the way you're asking, but it might be better to write a custom Failure handler that delete/re-enqueues jobs as they fail.

You can find the base failure backend here and an implementation that logs failed jobs to the Hoptoad exception tracking service here.

For example:

module Resque
  module Failure
    class RemoveRunaways < Base
      def save
        i=0
        while job = Resque::Failure.all(i)
          # Selectively remove all MyRunawayJobs from failure queue whenever they fail
          if job.fetch('payload').fetch('class') == 'MyRunawayJob'
            remove(i) 
          else
            i = i + 1
          end
        end
      end
    end
  end
end

EDIT: Forgot to mention how to specify this backend to handle Failures.

In your Resque initializer (eg: config/initializers/resque.rb):

# Use Resque Multi failure handler: standard handler and your custom handler
Resque::Failure::Multiple.classes = [Resque::Failure::Redis, Resque::Failure::RemoveRunaways]
Resque::Failure.backend = Resque::Failure::Multiple