Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Sidekiq/Airbrake only post exception when retries extinguished

I would like Airbrake to only be notified of errors when the retries are exhausted, but I can't seem to think of a way to implement it...

I can add a sidekiq_retries_exhausted hook to send the error to AirBrake but the only way I can think of catching the actual failures is to add a middleware that swallows the error, but then, the job will be marked as a success if there is no error... then there will never be any retries..

Hope that makes sense!

like image 750
Michael Baldry Avatar asked Oct 30 '13 12:10

Michael Baldry


2 Answers

As shown here (not my code):

    Airbrake.configure do |config|
      config.api_key = '...'
      config.ignore_by_filter do |exception_data|
        exception_data[:parameters] && 
        exception_data[:parameters]['retry_count'].to_i > 0
      end
    end
like image 29
Damien Roche Avatar answered Sep 21 '22 18:09

Damien Roche


I managed to implement this with a Sidekiq middleware that is inserted at the start of the list:

class RaiseOnRetriesExtinguishedMiddleware
    include Sidekiq::Util

  def call(worker, msg, queue)
    yield
  rescue Exception => e
    bubble_exception(msg, e)
  end

  private

  def bubble_exception(msg, e)
    max_retries = msg['retries'] || Sidekiq::Middleware::Server::RetryJobs::DEFAULT_MAX_RETRY_ATTEMPTS
    retry_count = msg['retry_count'] || 0
    last_try = !msg['retry'] || retry_count == max_retries - 1

    raise e if last_try
  end

  def retry_middleware
    @retry_middleware ||= Sidekiq::Middleware::Server::RetryJobs.new
  end
end

If its the last try and its thrown an exception, it'll let it bubble up (to Airbrake) otherwise it won't. This doesn't affect failure recording as that happens later in the chain.

like image 190
Michael Baldry Avatar answered Sep 19 '22 18:09

Michael Baldry