Run resque in background

Question

I have a working rails app with a resque queue system which works very well. However, I lack a good way of actually demonizing the resque workers.

I can start them just fine by going rake resque:work QUEUE="*" but I guess it's not the point that you should have your workers running in the foreground. For some reason nobody seems to adress this issue. On the official resque github page the claim you can do something like this:

PIDFILE=./resque.pid BACKGROUND=yes QUEUE="*" rake resque:work 

well - it doesn't fork into the background here at least.

Answer 1

A +1 for resque-pool - it really rocks. We use it in combination with God to make sure that it is always available.

# Resque God.watch do |w|    w.dir = RAILS_ROOT    w.name = "resque-pool"   w.interval = 30.seconds   w.start = "cd #{RAILS_ROOT} && sudo -u www-data sh -c 'umask 002 && resque-pool -d -E #{RAILS_ENV}'"   w.start_grace = 20.seconds   w.pid_file = "#{RAILS_ROOT}/tmp/pids/resque-pool.pid"    w.behavior(:clean_pid_file)    # restart if memory gets too high   #w.transition(:up, :restart) do |on|   #  on.condition(:memory_usage) do |c|   #    c.above = 350.megabytes   #    c.times = 2   #  end   #end    # determine the state on startup   w.transition(:init, { true => :up, false => :start }) do |on|     on.condition(:process_running) do |c|       c.running = true     end   end    # determine when process has finished starting   w.transition([:start, :restart], :up) do |on|     on.condition(:process_running) do |c|       c.running = true       c.interval = 5.seconds     end      # failsafe     on.condition(:tries) do |c|       c.times = 5       c.transition = :start       c.interval = 5.seconds     end   end    # start if process is not running   w.transition(:up, :start) do |on|     on.condition(:process_running) do |c|       c.running = false     end   end end 

This then gives you a really elegant way to reload code in your workers without interrupting jobs - simply kill -2 your resque-pool(s) when you deploy. Idle workers will die immediately, busy workers will die when they finish their current jobs, and God will restart resque-pool with workers using your new code.

These are our Resque tasks for Capistrano:

namespace :resque do    desc "Starts resque-pool daemon."   task :start, :roles => :app, :only => { :jobs => true } do     run "cd #{current_path};resque_pool -d -e #{rails_env} start"   end    desc "Sends INT to resque-pool daemon to close master, letting workers finish their jobs."   task :stop, :roles => :app, :only => { :jobs => true } do     pid = "#{current_path}/tmp/pids/resque-pool.pid"     sudo "kill -2 `cat #{pid}`"   end    desc "Restart resque workers - actually uses resque.stop and lets God restart in due course."   task :restart, :roles => :app, :only => { :jobs => true } do     stop # let God restart.   end    desc "List all resque processes."   task :ps, :roles => :app, :only => { :jobs => true } do     run 'ps -ef f | grep -E "[r]esque-(pool|[0-9])"'   end    desc "List all resque pool processes."   task :psm, :roles => :app, :only => { :jobs => true } do     run 'ps -ef f | grep -E "[r]esque-pool"'   end  end 

You might need to reconnect any DB connections when resque-pool forks workers - check the docs.

Answer 2

I had the same problem and the following works for me.

PIDFILE=./resque.pid BACKGROUND=yes QUEUE="*" rake resque:work >>  worker1.log & 

You can also redirect STDERR to the same log file.