Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Make MySQL Connector reuse a SSLSessionContextImpl

Tags:

java

mysql

Whilst investigating a suspected memory leak in our Java application, we found that much of the memory was being used by instances of SSLSessionContextImpl that were being created by the MySQL Connector - one instance per connection.

This became a problem for us, because as of OpenJDK 8u222, each instance of SSLSessionContextImpl contains a cache with an initial capacity of (20480 / 0.75) + 1 = 27307 (see JDK-8210985 and JDK source). This, combined with a Hikari connection pool that created ten new connections every half-hour (the default behaviour), meant that after a week we had about 1GB worth of dead SSLSessionContextImpl instances. These were in the tenured-generation memory pool, and phantom-referenced by the AbandonedConnectionCleanupThread class, and we were seeing our application die due to running out of memory (in a Docker container limited to 1GB), so my guess is that a proper GC could not be performed in time on the instances that had accumulated.

Naively, it seems as if the MySQL connector is not behaving quite right, and that for the same database server, it should be re-using the same SSLSessionContextImpl - after all, the JavaDoc of the SSLSessionContext interface (which it implements) says:

A SSLSessionContext represents a set of SSLSessions associated with a single entity. For example,it could be associated with a server or client who participates in many sessions concurrently.

That describes our scenario perfectly! Yet the Connector creates a new SSLSessionContext per connection.

So, two questions:

  1. Why does the MySQL connector not re-use the SSLContext? (Of course we can't read the developers' minds - but are there reasons why one might not want to re-use it?)
  2. Can we by some means make it re-use the SSLContext?
like image 758
PMah Avatar asked Apr 20 '20 16:04

PMah


1 Answers

Answer for question one:

The MariaDB Connector added functionality to create the SSLContext every time in CONJ-670 (released in 2.4.0, backported to 1.0.8.0). The reasoning was to allow the key store and trust store to dynamically change at runtime.

Workaround/mitigation

This doesn't force the SSLContext to be re-used - but it does reduce the memory footprint of having lots of them on the heap.

By adding a JVM flag -Djavax.net.ssl.sessionCacheSize=0, the pre-JDK-8210985 behaviour is restored.

Setting it to 0 means the cache size is effectively infinite. But in practice, that means it initially allocates a small amount of memory for the cache and allows it to grow indefinitely. Without this, the cache's initial capacity is 27307, which is rather large!

like image 162
PMah Avatar answered Nov 01 '22 01:11

PMah