How to deploy resque workers in production?

Question

The GitHub guys recently released their background processing app which uses Redis: http://github.com/defunkt/resque http://github.com/blog/542-introducing-resque

I have it working locally, but I'm struggling to get it working in production. Has anyone got a:

1. Capistrano recipe to deploy workers (control number of workers, restarting them, etc)
2. Deployed workers to separate machine(s) from where the main app is running, what settings were needed here?
3. gotten redis to survive a reboot on the server (I tried putting it in cron but no luck)
4. how did you work resque-web (their excellent monitoring app) into your deploy?

Thanks!

P.S. I posted an issue on Github about this but no response yet. Hoping some SO gurus can help on this one as I'm not very experienced in deployments. Thank you!

Answer 1

I'm a little late to the party, but thought I'd post what worked for me. Essentially, I have god setup to monitor redis and resque. If they aren't running anymore, god starts them back up. Then, I have a rake task that gets run after a capistrano deploy that quits my resque workers. Once the workers are quit, god will start new workers up so that they're running the latest codebase.

Here is my full writeup of how I use resque in production:

http://thomasmango.com/2010/05/27/resque-in-production

Answer 2

I just figured this out last night, for Capistrano you should use san_juan, then I like the use of God to manage deployment of workers. As for surviving a reboot, I am not sure, but I reboot every 6 months so I am not too worried.

Although he suggest different ways of starting it, this is what worked easiest for me. (Within your deploy.rb)

require 'san_juan' after "deploy:symlink", "god:app:reload" after "deploy:symlink", "god:app:start" 

To manage where it runs, on another server, etc, he covers that in the configuration section of the README.

I use Passenger on my slice, so it was relatively easy, I just needed to have a config.ru file like so:

require 'resque/server'  run Rack::URLMap.new \   "/" => Resque::Server.new 

For my VirtualHost file I have:

<VirtualHost *:80>         ServerName resque.server.com         DocumentRoot /var/www/server.com/current/resque/public          <Location />           AuthType Basic           AuthName "Resque Workers"           AuthUserFile /var/www/server.com/current/resque/.htpasswd           Require valid-user         </Location> </VirtualHost> 

Also, a quick note. Make sure you overide the resque:setup rake task, it will save you lots of time for spawning new workers with God.

I ran into a lot of trouble, so if you need any more help, just post a comment.