Rolling restart of process group in monit

Question

Does anyone have any suggestions of how I might go about achieving a rolling restart of a process group using monit?

Thanks in advance, fturtle

Answer 1

I'm not sure for which server you are talking about. But I can provide you an example for thin which supports rolling restart itself. (option onebyone: true)

So for monit you can use something like,

if ... then exec '/path/to/thin_restart.sh'

And thin_restart.sh would be something like,

source /path/to/scripts/rvm
rvm use your_gemset@some_ruby
thin -C thin.yml restart

And contents of thin.yml would look like,

port: 1337
pid: tmp/pids/thin.pid
rackup: /path/to/config.ru
daemonize: true
servers: 2
onebyone: true

There are other ways to fine tune this restarts based on pid. You can monitor files with pids and restart only those thin process based on conditions.

e.g.

check process app-1337
  with pid /path/to/app.1337.pid
  start = 'thin -d -p 1337 start'
  stop  = 'thin -d -p 1337 -P /path/to/thin.1337.pid stop'
  if cpu usage > 50% then restart
check process app-1338
  with pid /path/to/app.1338.pid
  start = 'thin -d -p 1338 start'
  stop  = 'thin -d -p 1338 -P /path/to/thin.1338.pid stop'
  if cpu usage > 50% then restart

The other way would be of using groups which monit provides.

Extending above example.

check process app-1337
  with pid /path/to/app.1337.pid
  group thin
  group thin-odd
  start = 'thin -d -p 1337 start'
  stop  = 'thin -d -p 1337 -P /path/to/thin.1337.pid stop'
  if cpu usage > 50% then restart
check process app-1338
  with pid /path/to/app.1338.pid
  group thin
  group thin-even
  start = 'thin -d -p 1338 start'
  stop  = 'thin -d -p 1338 -P /path/to/thin.1338.pid stop'
  if cpu usage > 50% then restart
check process app-1337
  with pid /path/to/app.1339.pid
  group thin
  group thin-odd
  start = 'thin -d -p 1339 start'
  stop  = 'thin -d -p 1339 -P /path/to/thin.1339.pid stop'
  if cpu usage > 50% then restart
check process app-1340
  with pid /path/to/app.1340.pid
  group thin
  group thin-even
  start = 'thin -d -p 1340 start'
  stop  = 'thin -d -p 1340 -P /path/to/thin.1340.pid stop'
  if cpu usage > 50% then restart

So now you can do following to restart all:

monit -g thin restart

or to achieve sort of rolling restart, restart odd ones then even. To restart only odd ones:

monit -g thin-odd restart

and to restart even:

monit -g thin-even restart