Advantages when using ActiveJob with Sidekiq compare with Sidekiq alone

Question

I'm reading some tutorials online that tell us using ActiveJob with Sidekiq. But I don't know why we should do that. I see that Sidekiq has all features that ActiveJob has.

Moreover, on Sidekiq document: here

Warning: by doing job retry through ActiveJob, you lose a lot of Sidekiq functionality:

1. Web UI visibility (the Retries tab will be empty)
2. You cannot iterate through retries with the Sidekiq::RetrySet API.
3. Sidekiq's log will not contain any failures or backtraces.
4. Errors will not be reported to Sidekiq's global error handlers
5. Many advanced Sidekiq features (e.g. Batches) will not work with AJ retries.

That is a signal somehow make me think that we shouldn't use Sidekiq with ActiveJob. Do I understand wrong about ActiveJob ? Are there any advantages when using ActiveJobs with sidekiq ?

Thanks

Answer 1

From rails ActiveJob guide

The main point is to ensure that all Rails apps will have a job infrastructure in place. We can then have framework features and other gems build on top of that, without having to worry about API differences between various job runners such as Delayed Job and Resque. Picking your queuing backend becomes more of an operational concern, then. And you'll be able to switch between them without having to rewrite your jobs.

basically what ActiveJob does is standardising the API interface for job queuer. This will help you to change from one job backend to the other with ease.

When you use Sidekiq with ActiveJob, you can benefit from the goodies sidekiq has to offer, but the real catch is when you find another queuer is best for your application, ActiveJob allows you to switch to your job queuer of choice with a one liner

# application.rb
config.active_job.queue_adapter = :sidekiq

Answer 2

As for me, the main feature of ActiveJob is support of GlobalId. Compare:

Sidekiq:

class SomeJob
  include Sidekiq::Worker
  def perform(record_id)
    record = Record.find(record_id)
    record.do_something
  end
end
SomeJob.perform_async(record.id)

ActiveJob:

class SomeJob < ApplicationJob
  def perform(record)
    record.do_something
  end
end

SomeJob.perform_later(record)

So convenient and much cleaner! 😍

About retries – yes, it's problem, and I don't know why the feature of giving to underlying system possibility to configure own params per job is overlooked in ActiveJob. So far as a workaround it's possible to use gem activejob-retry:

class SomeJob
  include ActiveJob::Retry.new(strategy: :exponential, limit: 0)
end

This disables ActiveJob's retries while Sidekiq's retries still work. It's possible to configure sidekiq's retries in config file by Sidekiq.default_worker_options['retry'] = 2

UPDATE

Retries issue is fixed in Sidekiq 6.0 🎉

Answer 3

Active Job is an abstraction layer on top of queueing technologies.

In theory, it allows you to program to a common interface, regardless of what you use underneath.

Sounds great right? Well, in practice, it can be much better to use Sidekiq directly.

• Most Ruby/Rails shops are all-in on Sidekiq. You'll never swap Sidekiq for some other queue tech, and in the unlikely event that you do, Active Job won't save you as much work as you think.
• Sidekiq and Active Job don't work well together. It's even worse in older versions. Sidekiq retries don't work out of the box with Active Job unless you're on Rails 6.0.2+ and Sidekiq 6.0.4. And if you're on a version of Rails before 5.1, retries won't work at all via Active Job unless you reinvent the wheel with Sidekiq middleware or adding yet another third party dependency that you don't really need.
• Active Job adds a lot of overhead, and that's just performance overhead captured in benchmarks. More importantly, there's also the cognitive overhead of having to understand how Active Job works in addition to understanding how Sidekiq works. More layers is more complexity, even if it doesn't look like it.
• You lose visibility on retries and errors, since Active Job has its own retry mechanism. They only become visible in Sidekiq once Active Job retries have exhausted and bubble up the exception to Sidekiq.
• The GlobalId feature that Lev mentions? That kind of magic leads to more detective work. Prefer explicit code, rather than hiding behaviour to save one line and misunderstand what the code is actually doing.