How to avoid DB2 driver Classloader Memory leak on Tomcat application .war file redeployment

Question

IBM's well supported JDBC driver creates a memory leak in combination with Tomcat's well supported connection pool. Please refer to Classloader memory leak on Tomcat application .war file redeployment.

java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [DB2JccConfiguration.properties]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
    at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading(WebappClassLoaderBase.java:1327)
    at org.apache.catalina.loader.WebappClassLoaderBase.getResource(WebappClassLoaderBase.java:1023)
    at com.ibm.db2.jcc.am.ud.run(Unknown Source)
    at java.security.AccessController.doPrivileged(AccessController.java:285)
    at com.ibm.db2.jcc.am.GlobalProperties.a(Unknown Source)
    at com.ibm.db2.jcc.am.GlobalProperties.d(Unknown Source)
    at com.ibm.db2.jcc.am.mq.run(Unknown Source)
    at java.util.TimerThread.mainLoop(Timer.java:567)
    at java.util.TimerThread.run(Timer.java:517)

I do not understand the suggested solution as it is in conflict with the most recommended practice of including the driver jar in the Tomcat lib directory.

We need shared deployment and re-deployment without Tomcat re-start. Please share your solution here if you have experience with this software combination and the described issue.

Answer 1

For driver version 4.22.29, I'm currently using this code in a ServletContextListener:

@Override
public void contextDestroyed(ServletContextEvent servletContextEvent) {
    // This fixes the JDBC driver not unloading corectly on a context reload  for DB2 JDBC 4.22.29
    try {
        logger.debug("Trying to stop the timer");
        new com.ibm.db2.jcc.am.iq() {
            // instance initializer to execute the fix when the anonymous class is instantiated, i.e. now
            {
                if (a != null) {
                    a.cancel();
                } else {
                    logger.debug("Timer is null, skipped");
                }
            }
        };
        logger.debug("Stopped the timer");
    } catch (Exception e) {
        logger.error("Could not stop the DB2 timer thread", e);
    }
}

Note: Since the DB2 driver JAR appears to be obfuscated, the timer storage (com.ibm.db2.jcc.am.iq.a) will probably be different for other driver versions. Also, the constructor of the class you're subclassing might have side effects, in my case there are none.

How I got to this solution

Get the exception

java.lang.NullPointerException
    at org.apache.catalina.loader.WebappClassLoaderBase.getResource(WebappClassLoaderBase.java:1600)
    at com.ibm.db2.jcc.am.wd.run(wd.java:49)
    at java.security.AccessController.doPrivileged(Native Method)
    at com.ibm.db2.jcc.am.GlobalProperties.a(GlobalProperties.java:146)
    at com.ibm.db2.jcc.am.GlobalProperties.d(GlobalProperties.java:100)
    at com.ibm.db2.jcc.am.dr.run(dr.java:124)  <------- point of interest <----------
    at java.util.TimerThread.mainLoop(Timer.java:555)
    at java.util.TimerThread.run(Timer.java:505)

The main class for the timer is com.ibm.db2.jcc.am.dr.

Using IntelliJ, I set a breakpoint in its constructor. Wait for the breakpoint to hit:

Go to where it's instantiated, in my case in GlobalProperties. Look on what timer it is scheduled.

Find a way to access iq.a: Since this is a static protected field, we can inherit from iq and from inside that class, access the static field of the parent class to call cancel() on a.

Answer 2

This is a confirmed bug in the IBM JDBC driver version 4.19 (timer task that cannot be disabled). The workaround is to downgrade to version 4.18.