Jboss datasource recovering after database re-start

Question

Closed connections still in the connection pool - why?

servlet-

public class Index extends HttpServlet {

    TimeZoneService timeZoneService;

    public void doGet(HttpServletRequest req, HttpServletResponse res) throws ServletException {
        WebApplicationContext ctx = WebApplicationContextUtils.getRequiredWebApplicationContext(getServletContext());
        timeZoneService = (TimeZoneService) ctx.getBean("timeZoneService");
        timeZoneService.loadAllTimeZones();
        System.out.println("Done");
    }
}

public interface TimeZoneService {
    void loadAllTimeZones();
}

public class TimeZoneServiceImpl implements TimeZoneService {

    private TimeZoneDao tzDao;
    private Map<Long, String> tzOid2JavaName = new HashMap<Long, String>();

    public void loadAllTimeZones() {
        List<TimeZone> timeZones = tzDao.findAllTimeZones();
        for (TimeZone tz : timeZones) {
            tzOid2JavaName.put(tz.getOid(), tz.getJavaName());
        }
    }

    public void setTzDao(TimeZoneDao tzDao) {
        this.tzDao = tzDao;
    }
}

public interface TimeZoneDao {
    List<TimeZone> findAllTimeZones() throws DataAccessException;  
}

public class TimeZoneDaoImpl extends JdbcDaoSupport implements TimeZoneDao {

    public List<TimeZone> findAllTimeZones() throws DataAccessException
    {
        StringBuffer sql = new StringBuffer();
        sql.append("SELECT TZ.OID, TZ.JAVA_NAME FROM TIME_ZONE TZ");
        List<TimeZone> timeZones = getJdbcTemplate().query(sql.toString(), new RowMapper() {
            public Object mapRow(ResultSet rs, int i) throws SQLException {
                TimeZone tz = new TimeZone();
                tz.setOid(rs.getLong("OID"));
                tz.setJavaName(rs.getString("JAVA_NAME"));
                return tz;
            }
        });

        return timeZones;
    }
}

public class TimeZone {
    private Long oid;
    private String javaName;

    public Long getOid() {
        return this.oid;
    }

    public void setOid(Long oid) {
        this.oid = oid;
    }

    public String getJavaName() {
        return this.javaName;
    }

    public void setJavaName(String javaName) {
        this.javaName = javaName;
    }
}

spring-config.xml

<beans>

    <jee:jndi-lookup id="dataSource" jndi-name="java:/OracleDS"/>

    <bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
        <property name="dataSource" ref="dataSource"/>
    </bean>

    <bean id="timeZoneDao" class="dao.impl.TimeZoneDaoImpl">
        <property name="dataSource" ref="dataSource"/>
    </bean>

    <bean id="timeZoneService" class="logic.impl.TimeZoneServiceImpl">
        <property name="tzDao" ref="timeZoneDao"/>
    </bean>

</beans>

web.xml

<web-app>

    <display-name>Spring</display-name>

    <context-param>
        <param-name>contextConfigLocation</param-name>
        <param-value>
            WEB-INF/spring-config.xml,classpath*:/META-INF/spring-config.xml</param-value>
    </context-param>

    <listener>
        <listener-class>
            org.springframework.web.context.ContextLoaderListener
        </listener-class>
    </listener>

    <servlet>
        <servlet-name>index</servlet-name>
        <display-name>Index page</display-name>
        <description>Landing page</description>
        <servlet-class>servlet.Index</servlet-class>
    </servlet>

    <servlet-mapping>
        <servlet-name>index</servlet-name>
        <url-pattern>/index</url-pattern>
    </servlet-mapping>

    <!-- Session Timeout (in minutes) -->
    <session-config>
        <session-timeout>60</session-timeout>
    </session-config>
</web-app>

mysql-ds.xml

<datasources> 
   <local-tx-datasource> 
      <jndi-name>OracleDS</jndi-name> 
      <connection-url>jdbc:mysql://localhost:3306/spring</connection-url> 
      <driver-class>com.mysql.jdbc.Driver</driver-class> 
      <user-name>spring_test</user-name> 
      <password>spring_test13</password> 
      <min-pool-size>1</min-pool-size> 
      <max-pool-size>5</max-pool-size> 
      <idle-timeout-minutes>2</idle-timeout-minutes> 
   </local-tx-datasource> 
</datasources>

Answer 1

Ok. Hope the following is useful for someone :-)

There is a datasource configuration setting - exception-sorter-class-name

According to Jboss this is used for a class that looks at vendor specific messages to determine whether sql errors are fatal and thus the connection should be destroyed. If none specified, no errors will be treated as fatal.

If using Oracle database this configuration is set to org.jboss.resource.adapter.jdbc.vendor.OracleExceptionSorter. This class has all the error codes that need to be treated as fatal and therefore connection needs to be destroyed.

In Jboss 4, error codes 17002 (connection reset) && 17008 (connection closed) are not included. They are added in Jboss 5. So, if you are using Jboss 4 and wondering why the connections are not being recovered, try adding the missing codes.

Answer 2

Here is connection validaton configuration with oracle DB for jboss 7.1+:

<validation>
   <valid-connection-checker class-name="org.jboss.jca.adapters.jdbc.vendor.OracleValidConnectionChecker"/>
   <check-valid-connection-sql>select 1 from dual</check-valid-connection-sql>
   <stale-connection-checker class-name="org.jboss.jca.adapters.jdbc.vendor.OracleStaleConnectionChecker"/>
   <exception-sorter class-name="org.jboss.jca.adapters.jdbc.extensions.oracle.OracleExceptionSorter"/>
</validation>

Now Jboss will validate your every connection.

Answer 3

This is a common issue that comes with using a connection pool. When the application borrows a connection from the pool, should the pool itself "test" the connection, to make sure it's still valid, or should it leave that up to the application?

If the pool tests the connection, this inevitably involves sending something down the connection to the database server (usually a basic SELECT of some kind). On high traffic systems, this is enormously wasteful, and can add considerable stress to the database server.

On low-traffic sites, though, where your database can handle the extra load, you can configure your datasource to make JBoss validate the connection before passing it to your application. If the connection is dead, JBoss will remove it from the pool and get a new one, so that will survive a database restart.

Any, add this to your mysql-ds.xml file:

<check-valid-connection-sql>select 1 from mytable</check-valid-connection-sql>

You have to pick the query yourself, make sure it's not an expensive one, because it'll be run a lot.

See the JBoss documentation wiki to see how to modify these datasource files.