Occasional NullPointerException in ResultSetImpl.checkColumnBounds or ResultSetImpl.getStringInternal

Question

First of all, please don't close this as duplicate of What is NullPointerException and how to fix it. I know what is NullPointerException, and I know how to solve it in my own code, but not when it is thrown by mysql-connector-java-5.1.36-bin.jar, which I have no control over.

We encountered the exception when running a common DB query, on a mySQL DB, that works most of the time. We started seeing this exception after deploying a new version, but the query in which it occurs didn't change in a long time.

Here's how the query looks like (with some necessary simplifications). I surrounded it with some of the logic performed before and after. The actual code is not all in a single method, but I put it in a single block to make it easier to understand.

Connection conn = ... // the connection is open
...
for (String someID : someIDs) {
    SomeClass sc = null;
    PreparedStatement
        stmt = conn.prepareStatement ("SELECT A, B, C, D, E, F, G, H FROM T WHERE A = ?");
    stmt.setString (1, "someID");
    ResultSet res = stmt.executeQuery ();
    if (res.next ()) {
        sc = new SomeClass ();
        sc.setA (res.getString (1));
        sc.setB (res.getString (2));
        sc.setC (res.getString (3));
        sc.setD (res.getString (4));
        sc.setE (res.getString (5));
        sc.setF (res.getInt (6));
        sc.setG (res.getString (7));
        sc.setH (res.getByte (8)); // the exception is thrown here
    }
    stmt.close ();
    conn.commit ();
    if (sc != null) {
        // do some processing that involves loading other records from the
        // DB using the same connection
    }
}
conn.close();

res.getByte(8) causes a NullPointerException with the following call stack :

com.mysql.jdbc.ResultSetImpl.checkColumnBounds(ResultSetImpl.java:763) com.mysql.jdbc.ResultSetImpl.getStringInternal(ResultSetImpl.java:5251) com.mysql.jdbc.ResultSetImpl.getString(ResultSetImpl.java:5173) com.mysql.jdbc.ResultSetImpl.getByte(ResultSetImpl.java:1650) org.apache.tomcat.dbcp.dbcp2.DelegatingResultSet.getByte(DelegatingResultSet.java:206) org.apache.tomcat.dbcp.dbcp2.DelegatingResultSet.getByte(DelegatingResultSet.java:206)

I searched for the source code of the relevant mysql-connector version and found this (taken from here):

756 protected final void checkColumnBounds(int columnIndex) throws SQLException {
757     synchronized (checkClosed().getConnectionMutex()) {    
758         if ((columnIndex < 1)) {    
759             throw SQLError.createSQLException(    
760                   Messages.getString("ResultSet.Column_Index_out_of_range_low",    
761                   new Object[] { Integer.valueOf(columnIndex), Integer.valueOf(this.fields.length) }), SQLError.SQL_STATE_ILLEGAL_ARGUMENT,
762                   getExceptionInterceptor());   
763         } else if ((columnIndex > this.fields.length)) {    
764             throw SQLError.createSQLException(    
765                   Messages.getString("ResultSet.Column_Index_out_of_range_high",    
766                   new Object[] { Integer.valueOf(columnIndex), Integer.valueOf(this.fields.length) }), SQLError.SQL_STATE_ILLEGAL_ARGUMENT,   
767                   getExceptionInterceptor());
768         }
769
770         if (this.profileSql || this.useUsageAdvisor) {
771             this.columnUsed[columnIndex - 1] = true;
772         }
773     }
774 }

As you can see, the exception occurs on this line :

} else if ((columnIndex > this.fields.length)) {   

which means this.fields somehow became null.

The closest thing I could find was this question, which has no answer.

I suspect the issue is not in the query I posted. Perhaps something went wrong with the Connection instance due to some other statements we are running on the same connection. All I can say is that we close every statement right after we execute it and read the data from its ResultSet.

EDIT (1/19/2017):

I couldn't recreate the error in my development environment. I thought it might be some mysql-connector bug triggered when the same connection is used for a long time. I limited the above loop to load at most 6 elements at a time. In addition, we upgraded the mysql-connector version to 5.1.40.

We still see NullPointerExceptions in ResultSetImpl, but this time at a different location.

The stack trace is :

com.mysql.jdbc.ResultSetImpl.getStringInternal(ResultSetImpl.java:5294) com.mysql.jdbc.ResultSetImpl.getString(ResultSetImpl.java:5151) org.apache.tomcat.dbcp.dbcp2.DelegatingResultSet.getString(DelegatingResultSet.java:198) org.apache.tomcat.dbcp.dbcp2.DelegatingResultSet.getString(DelegatingResultSet.java:198)

which implies this time the exception was thrown as a result of one of our res.getString() calls (I don't know which one).

I found the source for mysql-connector-java-5.1.40-bin.jar here and the relevant code is:

5292            // Handles timezone conversion and zero-date behavior
5293
5294            if (checkDateTypes && !this.connection.getNoDatetimeStringSync()) {
5295                switch (metadata.getSQLType()) {

which implies this.connection is null. this.connection is an instance variable of type MySQLConnection, which gets initialized in the constructors of ResultSetImpl and is only set to null when public void realClose(boolean calledExplicitly) is called (it is called by public void close(), which, according to the documentation within the source, is called when ResultSet.close() is called). We are definitely not closing the ResultSet prior to reading all the data from it.

Any ideas how to proceed?

Answer 1

It has been a long time since I posted this question, and I want to post an answer that describes the exact scenario that led to this tricky NullPointerException.

I think this may help future readers who encounter such a confusing exception think outside the box, since I had almost every reason to suspect this was a mysql connector bug, even though it wasn't after all.

While investigating this exception I was sure that my application can't possibly be closing the DB connection while trying to read data from it, since my DB connections are not shared across threads, and if the same thread closed the connection and then tried to access it, a different exception should have been thrown (some SQLException). That was the main reason I suspected a mysql connector bug.

It turned out that there were two threads accessing the same connection after all. The reason this was hard to figure out was that one of these threads was a garbage collector thread.

Going back to the code I posted:

Connection conn = ... // the connection is open
...
for (String someID : someIDs) {
    SomeClass sc = null;
    PreparedStatement
        stmt = conn.prepareStatement ("SELECT A, B, C, D, E, F, G, H FROM T WHERE A = ?");
    stmt.setString (1, "someID");
    ResultSet res = stmt.executeQuery ();
    if (res.next ()) {
        sc = new SomeClass ();
        sc.setA (res.getString (1));
        sc.setB (res.getString (2));
        sc.setC (res.getString (3));
        sc.setD (res.getString (4));
        sc.setE (res.getString (5));
        sc.setF (res.getInt (6));
        sc.setG (res.getString (7));
        sc.setH (res.getByte (8)); // the exception is thrown here
    }
    stmt.close ();
    conn.commit ();
    if (sc != null) {
        // do some processing that involves loading other records from the
        // DB using the same connection
    }
}
conn.close();

The problem lies in the "do some processing that involves loading other records from the DB using the same connection" section, which, unfortunately, I didn't include in my original question, since I didn't think the problem was there.

Zooming into that section, we have:

if (sc != null) {
    ...
    someMethod (conn);
    ...
}

And someMethod looks like this:

public void someMethod (Connection conn) 
{
    ...
    SomeOtherClass instance = new SomeOtherClass (conn);
    ...
}

SomeOtherClass looks like this (of course I'm simplifying here):

public class SomeOtherClass
{
    Connection conn;

    public SomeOtherClass (Connection conn) 
    {
        this.conn = conn;
    }

    protected void finalize() throws Throwable
    { 
        if (this.conn != null)
            conn.close();
    }

}

SomeOtherClass may create its own DB connection in some scenarios, but can accept an existing connection in other scenarios, like the one we have here.

As you can see, That section contains a call to someMethod that accepts the open connection as argument. someMethod passes the connection to a local instance of SomeOtherClass. SomeOtherClass had a finalize method which closes the connection.

Now, after someMethod returns, instance becomes eligible for garbage collection. When it is garbage collected, its finalize method is called by the garbage collector thread, which closes the connection.

Now we get back to the for loop, which continues to execute SELECT statements using the same connection which may be closed any time by the garbage collector thread.

If the garbage collector thread happens to close the connection while the application thread is in the middle of some mysql connector method that relies on the connection to be open, a NullPointerException may occur.

Removing the finalize method solved the issue.

We don't often override the finalize method in our classes, which made it very difficult to locate the bug.

Answer 2

The only way this could be happening, as far as I can see, is someone/something is calling ResultSet.close() right after ResultSet.getString() has been called, but before it returns.

This could be your application (is it multi-threaded, or does it have a tricky processing path), or you could be losing the connection sometime during this time, which causes the driver to close the connection implicitly, which closes all open statements, which closes all open result sets.

You shouldn't be getting an NPE though, we should raise a nicer SQLException.

You might check the mysqld error log to see if anything is going on there that implies that the connection is being lost?

-- http://bugs.mysql.com/bug.php?id=41484

Answer 3

It would have been helpful if you would have posted full stack trace.

It can be a concurrency issue if rs is a class variable where two threads are sharing your Resultset and executing it simultaneously for different queries.

One way to proceed on this is to use debugger with break point way before exception occurs and step through to notice where this happens. Put a breakpoint in a must use sql connector code too like connection.createStatement(...) or inside api, if it is a multithreading issue debugger will jump to it where you may analyse further by looking at call stack.

Answer 4

As pointed out in other answers, the root cause of NullPointerException is that ResultSet instance is getting closed. Below could be the possible reasons:

• Explicit close of result set by calling rs.close(). (it does not seems applicable here, as in above code rs.close() not called).
• Implicit close of result set. A result set can get closed implicitly by JDBC driver in some scenarios like:
  • If same PreparedStatement object is shared by multiple threads concurrently. Re-execution of query would implicitly close the previously opened ResultSet.

[Assuming your query is getting executed concurrently], It seems connection.preapresStatement is returning same statement object (incorrectly) when called concurrently.

What I can suggest you to try disabling the prepare statement cache (via cachePrepStmts connection property) and see if the problem still persists.