Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Memory leak on Tomcat 7 with Oracle JDBC drivers 12c - oracle.jdbc.driver thread failed to stop

Tags:

java

tomcat

jdbc

I have a web app deployed to Tomcat 7.0.54 that uses a datasource to connect to an Oracle 11g database. The datasource is configured in META-INF/context.xml and I've placed ojdbc7.jar in <tomcat-install-dir>/lib. I use a JNDI lookup to retrieve the datasource which I store in a singleton so that each DAO class can use it.

Everything works as expected, however when I undeploy the application (via Tomcat manager app) I see in the logs:

Oct 03, 2014 3:06:55 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/myapp] appears to have started a thread named [oracle.jdbc.driver.BlockSource.ThreadedCachingBlockSource.BlockReleaser] but has failed to stop it. This is very likely to create a memory leak.
Oct 03, 2014 3:06:57 PM org.apache.catalina.startup.HostConfig undeploy
INFO: Undeploying context [/myapp]

When I debug I can see this thread gets created as soon as the database is accessed (via the datasource).

My datasource config:

<Context antiResourceLocking="false">
    <Resource name="jdbc/myapp" auth="Container" 
        type="javax.sql.DataSource" driverClassName="oracle.jdbc.OracleDriver" 
        maxActive="20" maxIdle="10" maxWait="-1"
        username="myuser" password="mypass"
        url="jdbc:oracle:thin:@myserver:1521:mysid"
        removeAbandoned="true" removeAbandonedTimeout="10" logAbandoned="true"
        validationQuery="SELECT 1 FROM DUAL"   
        testOnBorrow="true" testOnReturn="true" testWhileIdle="true"  
        timeBetweenEvictionRunsMillis="1800000" numTestsPerEvictionRun="3"  
        minEvictableIdleTimeMillis="1800000"
    />
</Context>

EDIT

Further investigation has revealed that the issue occurs whether or not the datasource is accessed during application (or servlet) initialisation.

In actual fact, the problematic thread is only created, and thus the issue only exists, when using the 12c versions of Oracle's JDBC drivers (either ojdbc6.jar or ojdbc7.jar).

If I revert to using the 11.2.0.4 version of ojdbc6.jar the thread is never created and the memory leak warning never appears.

Should I downgrade JDBC driver (as suggested in https://stackoverflow.com/a/9177263/4105953)?

like image 446
finchie Avatar asked Oct 03 '14 14:10

finchie


People also ask

Is there a problem with Oracle JDBC driver in Tomcat?

This problem doesn't happen only with Oracle JDBC driver (oracle.jdbc.driver.BlockSource.ThreadedCachingBlockSource.BlockReleaser). If any of the 3rd party dependency starts a daemon thread for carrying out some task, then while stopping the Tomcat logs Warning message for them also. Also, this problem exists with Tomcat 8.5.X versions.

What does the JDBC driver severe mean?

SEVERE: A web application registered the JBDC driver [oracle.jdbc.driver.OracleDriver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.

Does redeploying create a fixed size memory leak?

The conclusion is that it creates a "fixed size memory leak", i.e. subsequent redeploys will not increase the memory leak. I do not have Oracle Support access, but the bug ID mentioned in the discussion is 16841748 (May 2013, might be solved now).

Why is my Tomcat server restarting after hot deployment?

This is normal, it happens when you hot deploy in Tomcat. It will not cause you any problems normally in production because you don't generally keep hot deploying updates in production, you just stop and restart the server instead. Show activity on this post.


1 Answers

I found a lengthy discussion about the subject here. The conclusion is that it creates a "fixed size memory leak", i.e. subsequent redeploys will not increase the memory leak.
I do not have Oracle Support access, but the bug ID mentioned in the discussion is 16841748 (May 2013, might be solved now).

A possible workaround is to actually use the datasource once (get connection, do dummy query, close connection) when Tomcat is started via a custom servlet that is configured to "load-on-startup" in tomcat/conf/web.xml. This should start the Oracle driver thread(s) (see also the FAQ about driver threads) outside of the scope of the class-loader of your web-app, thus preventing the "fixed size memory leak".

Note that a similar issue exists for the MySQL JDBC driver but has a decent solution. Such a solution may exist for a recent version of the Oracle JDBC driver (I don't know).

like image 148
vanOekel Avatar answered Oct 18 '22 18:10

vanOekel