Huge memory leak / bloat after Upgrading from Rails 3.0 to Rails 3.2 (Ruby 1.9.3 + Passenger)

Question

Last week we upgraded a small project which was running on Rails 3.0.1 to Rails 3.2.2.

Shortly after the upgrade we recognized that occasionally, 2-3 times a day, we're seening a huge Phusion Passenger process (1-5 GB).

We're running Phusion Passenger 3.0.11 and Ruby 1.9.3-p0. We already tried different Ruby version (1.9.2-p290 and 1.9.3-p125) without a success.

Afterwards, we tried to track our memory usage with Oink. Unfortunately Oink doesn't show the reason for the memory bloat - The large processes seem to not write to the logfiles anymore.

When we downgraded back to Rails 3.0.1 the problem is gone. Does anyone have similar problems? We really checked our code for possible memory leaks, such as too many ActiveRecord instances, but didn't find any.

Do you think it's worth it to try Unicorn instead of Passenger? Or is it likely that we'll run into the same problem?

Any suggestions about how to trace the memory leak are welcome. We already set up newrelic, but it doesn't show detailed information about the memory leak.

Answer 1

If this is Red Hat compatible Linux, you can use SystemTap. I am not sure if SystemTap is available on Debian/Ubuntu systems, if not there is alternative called DTrace. Here are some articles - I have been pretty successful with tracking down several regressions, although none of these were memory issues (maybe you can find a STP script that could do the trick for you). Read here:

http://lukas.zapletalovi.com/2012/02/peek-into-your-ruby-app-with-systemtap.html http://lukas.zapletalovi.com/2012/01/probing-ruby-apps-with-systemtap-in.html http://sourceware.org/systemtap/wiki/RubyMarker

The last link shows probes you can hoop into in SystemTaps in Ruby. There are thinks like gc runs or memory allocations, could help you. Good luck!