Background jobs with Resque on Heroku

Question

I am having a really strange problem on Heroku that I have been spinning my wheels to figure out for a while now.

My app has a few external API calls and mailers which I have set up ActiveJob to run in the background. On Heroku I have two workers set up as and I am using a Resque/Redis combo for the jobs as per the below snippets. I am using the Redis Cloud add-on on Heroku.

Config / setup

Procfile

web: bundle exec puma -C config/puma.rb
resque: env TERM_CHILD=1 QUEUE=* bundle exec rake resque:work

lib/tasks/resque.rake

require "resque/tasks"
require "resque/scheduler/tasks"

task "resque:setup": :environment do
  Resque.before_fork = proc { ActiveRecord::Base.connection.disconnect! }
  Resque.after_fork = proc { ActiveRecord::Base.establish_connection }
end

config/initializers/active_job.rb

Rails.application.config.active_job.queue_adapter = :resque

config/initializers/redis.rb

if ENV["REDISCLOUD_URL"]
  $redis = Redis.new(url: ENV["REDISCLOUD_URL"])
end

config/initializers/resque.rb

if Rails.env.production?
  uri = URI.parse ENV["REDISCLOUD_URL"]
  Resque.redis = Redis.new(host: uri.host, port: uri.port,
                           password: uri.password)
else
  Resque.redis = "localhost:6379"
end

The problem

The problem I am having is when a user is using the app in browser (i.e., interfacing with the web worker) and performs an action which triggers one of the ActiveJob jobs the job is run "inline" using the web worker and not the resque worker. When I run the specific model method that queues the job in my Heroku app console (opened by running heroku run rails console) it adds the job to Redis and runs it using the resque worker as expected.

Why would one way work properly and the other way not work? I have looked at almost every tutorial / SO question on the topic and have tried everything so any help getting the jobs to be run but the right worker would be amazing!

Thanks in advance!

Answer 1

I managed to solve the problem by playing with my config a little. It seems that actions were being tunnelled through ActiveJob's "Inline" default rather than via Resque. To get things working I just had to direct Resque.redis to be equal to the $redis variable set in config/initializers/redis.rb so everything was pointing to the same Redis instance and then move the config set in config/initializers/active_job.rb to application.rb.

For reference, the new & improved config that all works is:

Config / setup

Procfile

web: bundle exec puma -C config/puma.rb
resque: env TERM_CHILD=1 RESQUE_TERM_TIMEOUT=7 QUEUE=* bundle exec rake resque:work

lib/tasks/resque.rake

require "resque/tasks"

task "resque:setup" => :environment

config/application.rb

module App
  class Application < Rails::Application
    ...

    # Set Resque as ActiveJob queue adapter.
    config.active_job.queue_adapter = :resque
  end
end

config/initializers/redis.rb

if ENV["REDISCLOUD_URL"]
  $redis = Redis.new(url: ENV["REDISCLOUD_URL"])
end

config/initializers/resque.rb

Resque.redis = Rails.env.production? ? $redis : "localhost:6379"

Answer 2

thanks a lot for providing the answer. It saved me a lot of time.

You have one typo inside your Procfile.

It should be resque instead of rescue.

resque: env TERM_CHILD=1 RESQUE_TERM_TIMEOUT=7 QUEUE=* bundle exec rake resque:work

Also, I had to type in one more command to get this all to work in production. Hopefully this helps someone.

heroku ps:scale resque=1 --app appname

This command scales the resque process to 1 dyno(free). You can also do this from the dashboard on heroku.

You can read more about it on the heroku docs https://devcenter.heroku.com/articles/scaling