Analysis of 90% threads in java.lang.Thread.State: WAITING (parking)

Question

Thread count in my tomcat application server is growing every day.

When I have taken thread dump for analysis.

I found that out of 430 threads 307 threads are with this status.

Sample stacktrace

"pool-283-thread-1" #2308674 prio=5 os_prio=0 tid=0x000000000a916800 nid=0x1101 waiting on condition [0x00002aec87f17000]
   java.lang.Thread.State: WAITING (parking)
    at sun.misc.Unsafe.park(Native Method)
    - parking to wait for  <0x00000006d9929ec0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

   Locked ownable synchronizers:
    - None

"pool-282-thread-1" #2307106 prio=5 os_prio=0 tid=0x000000000a4fb000 nid=0x78e3 waiting on condition [0x00002aec87e16000]
   java.lang.Thread.State: WAITING (parking)
    at sun.misc.Unsafe.park(Native Method)
    - parking to wait for  <0x00000006d8ca7bf8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

   Locked ownable synchronizers:
    - None

Environment

JDK: jdk1.8.0_60
OS: Linux
Tomcat: tomcat-7.0.65

Not sure if this is causing the issue.

Appreciate any help on this.

Answer 1

This is typical resource leak. You are using some sort of ExecutorService somewhere in your application and you are not closing that pool after work is done causing threads to await forever.

You should call ExecutorService#shutdown() to close pool and release/terminate its threads after work is done.

Threads names like pool-282-thread-1 pool-283-thread-1 suggests that you are most probably using single thread pool executor (because pool no. is large and thread no. is only 1). The idea behind ExecutorService is to reuse threads that are idle to do some more work. So insteed of createing new ExecutorService each time you need to do some background work, you should rather share single instance and use it in your application.