Ways to understand why a ruby process block at 100% CPU

Question

We have a big rails app and since a few days, one by one, our ruby process seems to block in a loop and eat 100% of the CPU until the passenger server die raising 502 erros.

Do you know the best ways to figure that out why?

I've tried New Relic but it's only performance stuff, and our errors are too numerous to try to guess what is the problem. (We have a lot of request a day and lots of UTF-8 BSON Errors because we are using UTF-8 urls)

Using:

• Rails 3.2.6 with Ruby 1.9.2p290
• Passenger 3.0.13
• MongoDB 2.0.1 with Mongoid 2.4.11
• Nginx
• FreeBSD 8.2

Answer 1

A handy way to find out where your ruby is stuck is to attach gdb to the running process and call rb_backtrace(). That will print the current stack trace to stderr, which is likely directed to a log file. Here's a short walk through I found on a blog. The stack trace should help you localize where your process is stuck and allow you to figure out what's going on via code inspection or by adding logging instrumentation around the critical code path.

Aman Gupta has a gdb to ruby hooks library gdb.rb I've found handy at times.