Rails, ActiveJobs and AWS SQS: what happen to my jobs when a worker instance is killed?

Question

We are using Rails 5.0.2 and ActiveJobs in AWS Beanstalk with SQS as a backend with the gem active_elastic_job.

We have a job defined like this:

class MyJob < ActiveJob::Base
  rescue_from(StandardError) do |exception|
    self.class.set(:wait => 1.minutes).perform_later
  end

  def perform
    MyLongTask.run
  end
end

We experienced that when one instance in the worker environment is killed (due auto-scaling or something) the rescue_from is not executed and the Job is not sent back to the Queue.

How can we capture the moment the instance is been called to be killed so we can gracefully react and wrapping up before my process is really killed? (if possible)

Update

I am trying this

class MyJob < ActiveJob::Base
  def perform
    begin
      sleep(100)
    rescue SignalException => e
      # send signal to some log place
      raise e
    end
  end
end

But the log is never sent not when I terminate the instance neither when I kill the puma process with $ restart puma

Answer 1

By catching the SIGTERM signal, you can perform any cleanup you'd like before the process is terminated (your environment might send a SIGKILL later which cannot be caught).

Signal.trap(:SIGTERM) {
  # perform cleanup here
  exit
}