I've got a Rails 2.3 app that is keeping too many MySQL connections open. After less than a day (at ~400rpm) one process had 83 ESTABLISHED connections to the two mysql servers we use.
We're using the mysql2 gem (0.2.18), and the mysql client is: mysql Ver 14.12 Distrib 5.0.77, for redhat-linux-gnu (i686) using readline 5.1
.
How can I troubleshoot where these leaks are happening? In our testing, we're never able to leak connections, its only in production.
In MySQL, we can run show processlist;
to see the open connections. On the app server, we can count the number of connections per pid with sudo netstat -ntp | grep 3306 | grep ESTABLISHED | awk '{print $7}' | sort | uniq -c | sort -n
.
If clients encounter Too many connections errors when attempting to connect to the mysqld server, all available connections are in use by other clients. The permitted number of connections is controlled by the max_connections system variable. To support more connections, set max_connections to a larger value.
The MySQL “Too many connections” error occurs when more queries are sent to a MySQL database than can be processed. The error can be fixed by setting a new number of maximum connections in the configuration file or globally.
MySQL Connection Limits At provision, Databases for MySQL sets the maximum number of connections to your MySQL database to 200. You can raise this value by Changing the MySQL Configuration.
I fixed this by adding "wait_timeout: 300" to our database.yml. While that does close the unused mysql connections, it doesn't explain where they came from.
One random idea: fork the mysql2 gem, add some debugging into Mysql2::Client#initialize, and run your app as normal. You could print a few lines of the stack when the client is initialized, and track down who's causing the leak.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With