Memory leak in JDBC4Connection

Question

I'm trying to catch a memory leak in one of our Java daemons, and after dumping the memory and analyzing it via Memory Analyzer Tool, noticed that most of the leak is caused by the JDBC4Connection:

10 instances of "com.mysql.jdbc.JDBC4Connection", loaded by "sun.misc.Launcher$AppClassLoader @ 0x2aaab620ed00" occupy 858,283,752 (81.55%) bytes. Biggest instances:

* com.mysql.jdbc.JDBC4Connection @ 0x2aaab64ad820 - 87,110,160 (8.28%) bytes.
* com.mysql.jdbc.JDBC4Connection @ 0x2aaab64af520 - 86,730,408 (8.24%) bytes.
* com.mysql.jdbc.JDBC4Connection @ 0x2aaab64ad0e0 - 86,584,048 (8.23%) bytes.
* com.mysql.jdbc.JDBC4Connection @ 0x2aaab64aede0 - 86,488,800 (8.22%) bytes.
* com.mysql.jdbc.JDBC4Connection @ 0x2aaab61f5320 - 85,752,872 (8.15%) bytes.
* com.mysql.jdbc.JDBC4Connection @ 0x2aaab64ae6a0 - 85,603,280 (8.13%) bytes.
* com.mysql.jdbc.JDBC4Connection @ 0x2aaab64adf60 - 85,270,440 (8.10%) bytes.
* com.mysql.jdbc.JDBC4Connection @ 0x2aaab61f4be0 - 85,248,592 (8.10%) bytes.
* com.mysql.jdbc.JDBC4Connection @ 0x2aaab64afc60 - 85,120,704 (8.09%) bytes.
* com.mysql.jdbc.JDBC4Connection @ 0x2aaab61f5a60 - 84,374,448 (8.02%) bytes.

Keywords
com.mysql.jdbc.JDBC4Connection
sun.misc.Launcher$AppClassLoader @ 0x2aaab620ed00

I'm pretty sure that we closing all of the MySQL resources, but can't quite find what causing this.

Is there any good way to catch it? Have you experienced this in the past, and can advise what should I look for?

P.S.: Looking deeper with MAT, I see the following info:

com.mysql.jdbc.JDBC4Connection @ 0x2aaab64ad820 | 1,856 | 87,110,160 | 8.28% |- java.util.HashMap @ 0x2aaab62115a8 | 64 | 87,021,632 | 8.27% | '- java.util.HashMap$Entry[16384] @ 0x2aaae182e970| 131,096 | 87,021,568 | 8.27%

It seems as each JDBC holds a huge amount of Hashmap entries (>6000 objects), and does not free them at all.

Thanks in advance!

Answer 1

Duffymo is almost certainly right. In the past when we've had memory leaks, it's practically ALWAYS the MySQL JDBC driver. Just forgetting to close one little ResultSet or Connection or Statement somewhere. I ended up auditing the entire codebase for every time we used those to find the problem and ensure they get closed.

As for the HashMap, I've seen that too. I haven't looked at the source but my impression was that the MySQL driver stored the rows (at least row values) in HashMaps internally.

Leaking ResultSets is sadly easy. The idea of those closeable resources that take care of this themselves coming in JDK 7 or 8 really appeals to me for this reason.

You could insert a shim class somewhere (say for Connection) to log each opened/closed resource to see if you can catch where the leak is without directly reading all your source.