Nginx rolling restart of Rails app with capistrano

Question

For the life of me I can't figure out how to make this work properly.

The problem is similar to what others have, such as: How to do a rolling restart of a cluster of mongrels

We, however, are using Nginx/Passenger instead of Mongrel.

The issue is that on a deploy if we use this standard :restart task:

task :restart, :roles => [:app], :except => {:no_release => true} do
  run "cd #{deploy_to}/current && touch tmp/restart.txt"
end

It touches the restart.txt file across every web server, but any passenger instances currently serving requests need to finish before the new ones are spawned it seems. This creates a serious delay and causes our app to be unavailable for up to 2 minutes while everything is coming back up.

In order to get around that the plan is to do the following:

1. deploy code
2. go to server 1, remove it from the load balancer
3. restart nginx-passenger on server 1
4. wait 60 seconds
5. add server 1 back to load balancer
6. go to server 2 (repeat steps 3 - 5)

To accomplish this, I attempted this:

(lb.txt is the file that the load balancer looks for)

task :restart, :roles => [:app], :except => {:no_release => true} do
  servers = find_servers_for_task(current_task)
  servers.map do |s|
    run "cd #{deploy_to}/current && echo '' > public/lb.txt", :host => s.host
    run %Q{rvmsudo /etc/init.d/nginx-passenger restart > /dev/null}, :host => s.host
    sleep 60
    run "cd #{deploy_to}/current && echo 'ok' > public/lb.txt", :host => s.host
  end
end

This almost works, however, during the deploy it seemed to run the loop through the servers once per servers listed in the :app role. We currently have 6 app servers, so the loop runs 6 times, restarting nginx-passenger 6 times per server.

I just need this loop to run through one time.

I know it seems that eventually passenger will get rolling restarts, but they do not seem to exist yet.

If it helps, we are using Capistrano 2.x and Rails 3

Any help would be great.

Thanks.

Answer 1

run "cd #{deploy_to}/current && echo 'ok' > public/lb.txt", :host => s.host

should actually be:

run "cd #{deploy_to}/current && echo 'ok' > public/lb.txt", :hosts => s.host