Find memory leak in very complex Ruby app

Question

everyone!

It's nice to work with Ruby and write some code. But in past of this week, i notice that we have some problem in our application. Memory usage is growing like O(x*3) function.

Our application very complex, it is based on EventMachine and other external libs. Even more, it is running under amd64 bit version of FreeBSD using Ruby 1.8.7-p382

I'v tried to research by myself the way how find memory leak in our app. I've found many tools and libs, but they doesn't work under FreeBSD'64bit and I have no idea how step up to find leaks in huge ruby application. It's OK, if you have few files with 200-300 lines of code, but here you have around 30 files with average 200-300 line's of code.

I just realize, i need too much of time to find those leaks, doing stupid actions: believe/research/assume that some of part of this code is may be actually leaking and wrap some tracking code, like using ruby-prof gem technice. But it's so painfully slow way, because as i said we have too much of code.

So, my question is how to find memory leak in very complex Ruby app and not put all my life into this work?

Thx in advance

Answer 1

One thing to try, even though it can massively degrade performance, is to manually trigger the garbage collector by calling GC.start every so often. How often is kind of subjective, as the more you run it the slower the app, and the less you run it the higher the memory footprint.

For whatever reason, the garbage collector may go on vacation from time to time, presumably not wanting to interfere if there is some heavy processing going on. As such you may have to manually call to have your trash taken away.

One way to avoid creating trash is to use memory more efficiently. Don't create hashes when arrays will do the job, don't create arrays when a single string will suffice, and so on. It will be important to profile your application to see what kind of objects are cluttering up your heap before you just start hacking away randomly.

If you can, try and use 1.9.2 which has made significant gains in terms of memory management. Ruby Enterprise Edition is also an option if you need 1.8.7 compatibility, as it's essentially a better garbage collector for that version.

Answer 2

How hard would it be to run your app on a linux box? If you don't have the same memory problems there, it is probably something specific with your ruby runtime. If you do have the same problems, you can use all the tools and libs that are linux only.

Another alternative - can you wrap your unit tests with some memory tracking code? Most unit test frameworks make it easy to add some code before/after each test. Or you could just run each test 1000000000 times and see if the memory goes out of control? if it does, you know something that happens in that test is causing the leak, and you can continue to isolate the problem.