Weblogic Stuck Thread on JDBC call

Question

We frequently get a series of Stuck threads on our Weblogic servers. I've analyzed this over a period of time.

What I'd like to understand is whether this stuck thread block indicates it is still reading data from the open socket to the database since the queries are simple SELECT stuff?

at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.read(SocketInputStream.java:129)
        at oracle.net.ns.Packet.receive(Packet.java:239)
        at oracle.net.ns.DataPacket.receive(DataPacket.java:92)

We've run netstat and other commands, the sockets from the Weblogic app server to the Database match the number of connections in the pool.

Any ideas what else we should be investigating here?

Stack trace of thread dump:

"[STUCK] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon prio=10 tid=0x61a5b000 nid=0x25f runnable [0x6147b000..0x6147eeb0]
   java.lang.Thread.State: RUNNABLE
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.read(SocketInputStream.java:129)
        at oracle.net.ns.Packet.receive(Packet.java:239)
        at oracle.net.ns.DataPacket.receive(DataPacket.java:92)
        at oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:172)
        at oracle.net.ns.NetInputStream.read(NetInputStream.java:117)
        at oracle.net.ns.NetInputStream.read(NetInputStream.java:92)
        at oracle.net.ns.NetInputStream.read(NetInputStream.java:77)
        at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1023)
        at oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:999)
        at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:584)
        at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:183)
        at oracle.jdbc.driver.T4CStatement.fetch(T4CStatement.java:1000)
        at oracle.jdbc.driver.OracleResultSetImpl.close_or_fetch_from_next(OracleResultSetImpl.java:314)
        - locked <0x774546e0> (a oracle.jdbc.driver.T4CConnection)
        at oracle.jdbc.driver.OracleResultSetImpl.next(OracleResultSetImpl.java:228)
        - locked <0x774546e0> (a oracle.jdbc.driver.T4CConnection)
        at weblogic.jdbc.wrapper.ResultSet_oracle_jdbc_driver_OracleResultSetImpl.next(Unknown Source)

The bit starting from weblogic.work.ExecuteThread.run to here has been omitted. We have 8 sets of thread dumps - and each show the thread waiting on the same line, and the same object locked

at oracle.jdbc.driver.OracleResultSetImpl.close_or_fetch_from_next(OracleResultSetImpl.java:314)
        - locked <0x774546e0> (a oracle.jdbc.driver.T4CConnection)

Answer 1

At the time the stack was printed, it seems blocked waiting for more data from the server

at oracle.jdbc.driver.OracleResultSetImpl.next(OracleResultSetImpl.java:228)

Maybe it is just the query which is taking more than StuckThreadMaxTimeand WL issues a Warning.

If possible I would try:

• Find which query or queries are getting the threads stuck and check execution time
• Use Wireshark to analyze communication with database
• Have a look at the driver source code (JD comes to mind) to understand stack trace