Using god only to kill

Question

I serve my software using passenger. It spawns many ruby processes.

Sometimes one of these rubies becomes bloated and I want it to die.

I was hoping to use god to that intent. My idea was to monitor all these rubies and if it is consuming more than 500MB of memory for 3 cycles, god should try to gracefuly kill it. If it remains alive for more than 5 minutes then god should kill it not gracefully.

It seems to me that god always tries to run the service again, so it forces us to provide a start command. Is it possible to use god only to kill bad behaviored processes and let the passenger spawner to bring them back to live when necessary?

Answer 1

Answer to your question lies in question itself. you can kill ruby processes using god gem which is ruby process process monitor framework by github guys.

basically, here is how it works:

1. configure god to monitor process it can be anything from apache,passenger,mongrel or just simple file doing a long-running task.
2. Set conditionals in god's configuration file based upon which god will execute some predefined code.

here is a simple example(taken from docs). consider this as file long running process that runs undefiantly which we want to monitor for memory usage, lets call it simple.rb

loop do
  puts 'Hello'
  sleep 1
end

now, we install the god gem & configure it to as run as superuser so it can kill/spawn processes and next create a configuration file. example(also taken from docs):

God.watch do |w|
  w.name = "simple"
  w.start = "ruby /full/path/to/simple.rb"
  w.keepalive(:memory_max => 500.megabytes)
end

Here, as you may have got the idea if the process memory usage goes above 500 megabytes, god will restart it. here are few resources that might help, if you are getting started with process management using god gem:

• Example gist - Passenger worker monitor to kill workers which use too much RAM(Don't use god, but spawns a new passenger worker instead)
• Project Homepage
• Github Page
• An indepth tutorial using god with rails & passenger

Now, please remember ALL configuration for god is actually legal ruby code so you can get creative & do all sorts of things.

lastly, if you are frequently finding yourself running long running process, I advice you to try JRuby which is works much better with long running processes due to JVM & LOT faster than MRI

Answer 2

I use the same setup on many of my projects and had the same memory leaking issues. After messing around with monitoring, we decided to use the passenger features to tackle it. Specifically it allows the setting (e.g.) PassengerMaxRequests 300 which shuts down any instance when it has served that number of requests.

If you use it, make sure that PassengerMinInstances is set to 0 because it preceedes the setting for max requests.