Sidekiq transient vs fatal errors

Question

Is there a way to err from a Sidekiq job in a way that tells Sidekiq that "this error is fatal and unrecoverable, do not retry, send it straight to dead job queue"?

Looking at Sidekiq Error Handling documentation, it seems like it interpret all errors as transient, and will retry a job (if retry is enabled) regardless of the error type.

Answer 1

You should rescue those specific errors and not re-raise them.

def perform
  call_something
rescue CustomException
  nil
end

Edit:

Well, if you want to purposely send a message to the DLQ/DJQ, you'd need to make a method that does what #send_to_morgue does. I'm sure Mike Perham is going to come in here and yell at me for suggesting this but...

    def send_to_morgue(msg)
      Sidekiq.logger.info { "Adding dead #{msg['class']} job #{msg['jid']}" }
      payload = Sidekiq.dump_json(msg)
      now = Time.now.to_f
      Sidekiq.redis do |conn|
        conn.multi do
          conn.zadd('dead', now, payload)
          conn.zremrangebyscore('dead', '-inf', now - DeadSet.timeout)
          conn.zremrangebyrank('dead', 0, -DeadSet.max_jobs)
        end
      end
    end

The only difference you'd have to dig into what msg looks like going into that method but I suspect it's what normally hits the middleware before parse.

Answer 2

If found on GitHub a solution for your problem. In that post they suggested to write a custom middleware that handles the exceptions you want to prevent retries for. This is a basic example:

def call(worker, msg, queue)
  begin
    yield
  rescue ActiveRecord::RecordNotFound => e
    msg['retry'] = false
    raise
  end
end

You can extending that you get:

def call(worker, msg, queue)
  begin
    yield
  rescue ActiveRecord::RecordNotFound => e
    msg['retry'] = false
    raise
  rescue Exception => e
    if worker.respond_to?(:handle_error)
      worker.handle_error(e)
    else
      raise
    end
  end
end