Find the list of failed Sidekiq jobs with their arguments

Question

I am using Sidekiq for job management with a Rails APP. I would like to know how do you get the arguments of the failed jobs (Ids, Objects, error message, etc...)? On the WebUI, all you get is the nr of failed jobs. And if I understand correctly, the default is to add up all the times a given job failed. I have deployed my app and it is running on several workers. It is difficult to have to go over every single worker and try to find out, especially when you have a sidekiq.log that is many days old.

I search for an answer here and on the web. One of the closest was described on the folowing links.

How to find failed jobs list in sidekiq?

This allows to find out how many failed jobs I have over a period of time. However, I still do not know how to know exaclty what job failed and why.

Basically, I would like to somehow collect the job_ids and check periodically which ones have failed or if there is an easier way, just query Sidekiq/Redis and see what are the arguments and errors of the failed jobs.

I also visited this link: Get error message out of Sidekiq job

Here is an example job I am using

class ClassificationJob < ActiveJob::Base
 queue_as :default

 def perform(entity)
  entity.preprocess!
  entity.classify!
 end
end

I tried to modify to this

class ClassificationJob < ActiveJob::Base
 queue_as :default

 def perform(entity)
  entity.preprocess!
  entity.classify!
  store entity_id: entity.id.to_s

  entity_id = retrieve :entity_id
 end
end

But it spits the following error:

ArgumentError: You cannot include Sidekiq::Worker in an ActiveJob: ClassificationJob

Thanks,

Yannick

Answer 1

You are looking for retries. A "retry" is Sidekiq's term for a job which has failed and will be retried sometime in the future.

The Web UI uses the Sidekiq API documented on the API wiki page to list retries on the Retries tab:

https://github.com/mperham/sidekiq/wiki/API#retries

retries = Sidekiq::RetrySet.new
retries.each do |job|
  p [job.klass, job.args, job.jid]
end

Answer 2

I've just been searching for a way to narrow down different types of failures in the console from a given queue.

After a bit of digging,

To get a list of your workers:

query = Sidekiq::Failures::FailureSet.new.map(&:item)

query.map { |item| item['class'] }.uniq # Gives you a sense of what failed
query.map { |item| item['wrapped'] }.uniq # Gives you the job's original class
query.map { |item| item['queue'] }.uniq # Gives you a name of each of the queues
query.map { |item| item['args'] }.uniq # This would give you all the argument hashes for all failures.  

The args hash has the following keys:

job_class
job_id
queue_name
arguments
locale

I think you have to have this gem for it to work: https://github.com/mhfs/sidekiq-failures