SpringBootTest hangs following Java 21 migration

Question

Im trying to upgrage a SpringBoot 3.2.1/Java 17 application to Java 21. The app seems to be working fine but im facing an issue with all SpringBootTest tests. The test istself is successful but it seems to be stuck hanging after execution. I have extracted a thread dump and it looks like the snippet below.

Im thinking it may be related to Jndi-Dns-address-change-listener but im not sure and i haven't the faintest idea on how to solve this. Any ideas ?

Thanks for your help

"Attach Listener@10326" tid=0xc nid=NA runnable
  java.lang.Thread.State: RUNNABLE

"Jndi-Dns-address-change-listener@19314" tid=0x57 nid=NA runnable
  java.lang.Thread.State: RUNNABLE
      at sun.net.dns.ResolverConfigurationImpl.notifyAddrChange0(ResolverConfigurationImpl.java:-1)
      at sun.net.dns.ResolverConfigurationImpl$AddressChangeListener.run(ResolverConfigurationImpl.java:179)
      at java.lang.Thread.runWith(Thread.java:1596)
      at java.lang.Thread.run(Thread.java:1583)
      at jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:186)

"Notification Thread@1328" tid=0x2a nid=NA runnable
  java.lang.Thread.State: RUNNABLE

"Reference Handler@26419" tid=0x9 nid=NA runnable
  java.lang.Thread.State: RUNNABLE
      at java.lang.ref.Reference.waitForReferencePendingList(Reference.java:-1)
      at java.lang.ref.Reference.processPendingReferences(Reference.java:246)
      at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:208)

"Signal Dispatcher@26421" tid=0xb nid=NA runnable
  java.lang.Thread.State: RUNNABLE

"container-0@18025" tid=0x53 nid=NA sleeping
  java.lang.Thread.State: TIMED_WAITING
      at java.lang.Thread.sleep0(Thread.java:-1)
      at java.lang.Thread.sleep(Thread.java:509)
      at org.apache.catalina.core.StandardServer.await(StandardServer.java:564)
      at org.springframework.boot.web.embedded.tomcat.TomcatWebServer$1.run(TomcatWebServer.java:217)

"Catalina-utility-1@18010" tid=0x51 nid=NA waiting
  java.lang.Thread.State: WAITING
      at jdk.internal.misc.Unsafe.park(Unsafe.java:-1)
      at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:269)
      at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1758)
      at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
      at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
      at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1070)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
      at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
      at java.lang.Thread.runWith(Thread.java:1596)
      at java.lang.Thread.run(Thread.java:1583)

"Catalina-utility-2@18018" tid=0x52 nid=NA waiting
  java.lang.Thread.State: WAITING
      at jdk.internal.misc.Unsafe.park(Unsafe.java:-1)
      at java.util.concurrent.locks.LockSupport.park(LockSupport.java:371)
      at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(AbstractQueuedSynchronizer.java:519)
      at java.util.concurrent.ForkJoinPool.unmanagedBlock(ForkJoinPool.java:3780)
      at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3725)
      at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1707)
      at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)
      at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
      at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1070)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
      at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
      at java.lang.Thread.runWith(Thread.java:1596)
      at java.lang.Thread.run(Thread.java:1583)

"Common-Cleaner@26422" tid=0x1b nid=NA waiting
  java.lang.Thread.State: WAITING
      at jdk.internal.misc.Unsafe.park(Unsafe.java:-1)
      at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:269)
      at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1847)
      at java.lang.ref.ReferenceQueue.await(ReferenceQueue.java:71)
      at java.lang.ref.ReferenceQueue.remove0(ReferenceQueue.java:143)
      at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:218)
      at jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:140)
      at java.lang.Thread.runWith(Thread.java:1596)
      at java.lang.Thread.run(Thread.java:1583)
      at jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:186)

"Finalizer@26420" tid=0xa nid=NA waiting
  java.lang.Thread.State: WAITING
      at java.lang.Object.wait0(Object.java:-1)
      at java.lang.Object.<obsolete>(Object.java:-1)
      at java.lang.Object.<obsolete>(Object.java:-1)
      at java.lang.ref.NativeReferenceQueue.await(NativeReferenceQueue.java:48)
      at java.lang.ref.ReferenceQueue.remove0(ReferenceQueue.java:158)
      at java.lang.ref.NativeReferenceQueue.remove(NativeReferenceQueue.java:89)
      at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:173)

"HikariPool-1 housekeeper@12494" tid=0x4f nid=NA waiting
  java.lang.Thread.State: WAITING
      at jdk.internal.misc.Unsafe.park(Unsafe.java:-1)
      at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:269)
      at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1758)
      at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
      at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
      at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1070)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
      at java.lang.Thread.runWith(Thread.java:1596)
      at java.lang.Thread.run(Thread.java:1583)

"main@1" tid=0x1 nid=NA waiting
  java.lang.Thread.State: WAITING
      at java.lang.Object.wait0(Object.java:-1)
      at java.lang.Object.wait(Object.java:366)
      at java.lang.Thread.join(Thread.java:2078)
      at java.lang.Thread.join(Thread.java:2154)
      at java.lang.ApplicationShutdownHooks.runHooks(ApplicationShutdownHooks.java:114)
      at java.lang.ApplicationShutdownHooks$1.run(ApplicationShutdownHooks.java:47)
      at java.lang.Shutdown.runHooks(Shutdown.java:130)
      at java.lang.Shutdown.exit(Shutdown.java:167)
      - locked <0x5b0> (a java.lang.Class)
      at java.lang.Runtime.exit(Runtime.java:188)
      at java.lang.System.exit(System.java:1920)
      at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:56)

"OkHttp TaskRunner@25400" tid=0x9a nid=NA waiting
  java.lang.Thread.State: WAITING
      at java.lang.Object.wait0(Object.java:-1)
      at java.lang.Object.wait(Object.java:366)
      at java.lang.Object.wait(Object.java:488)
      at okhttp3.internal.concurrent.TaskRunner$RealBackend.coordinatorWait(TaskRunner.kt:294)
      at okhttp3.internal.concurrent.TaskRunner.awaitTaskToRun(TaskRunner.kt:218)
      at okhttp3.internal.concurrent.TaskRunner$runnable$1.run(TaskRunner.kt:59)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
      at java.lang.Thread.runWith(Thread.java:1596)
      at java.lang.Thread.run(Thread.java:1583)

"pool-4-thread-1@18840" tid=0x56 nid=NA waiting
  java.lang.Thread.State: WAITING
      at jdk.internal.misc.Unsafe.park(Unsafe.java:-1)
      at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:269)
      at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1758)
      at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
      at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
      at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1070)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
      at java.lang.Thread.runWith(Thread.java:1596)
      at java.lang.Thread.run(Thread.java:1583)

"pool-5-thread-1@25043" tid=0x98 nid=NA waiting
  java.lang.Thread.State: WAITING
      at jdk.internal.misc.Unsafe.park(Unsafe.java:-1)
      at java.util.concurrent.locks.LockSupport.park(LockSupport.java:371)
      at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(AbstractQueuedSynchronizer.java:519)
      at java.util.concurrent.ForkJoinPool.unmanagedBlock(ForkJoinPool.java:3780)
      at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3725)
      at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1707)
      at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
      at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1070)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
      at java.lang.Thread.runWith(Thread.java:1596)
      at java.lang.Thread.run(Thread.java:1583)

"pool-5-thread-2@25517" tid=0xa0 nid=NA waiting
  java.lang.Thread.State: WAITING
      at jdk.internal.misc.Unsafe.park(Unsafe.java:-1)
      at java.util.concurrent.locks.LockSupport.park(LockSupport.java:371)
      at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(AbstractQueuedSynchronizer.java:519)
      at java.util.concurrent.ForkJoinPool.unmanagedBlock(ForkJoinPool.java:3780)
      at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3725)
      at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1707)
      at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
      at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1070)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
      at java.lang.Thread.runWith(Thread.java:1596)
      at java.lang.Thread.run(Thread.java:1583)

"pool-5-thread-3@25746" tid=0xa1 nid=NA waiting
  java.lang.Thread.State: WAITING
      at jdk.internal.misc.Unsafe.park(Unsafe.java:-1)
      at java.util.concurrent.locks.LockSupport.park(LockSupport.java:371)
      at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(AbstractQueuedSynchronizer.java:519)
      at java.util.concurrent.ForkJoinPool.unmanagedBlock(ForkJoinPool.java:3780)
      at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3725)
      at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1707)
      at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
      at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1070)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
      at java.lang.Thread.runWith(Thread.java:1596)
      at java.lang.Thread.run(Thread.java:1583)

"pool-5-thread-4@25747" tid=0xa2 nid=NA waiting
  java.lang.Thread.State: WAITING
      at jdk.internal.misc.Unsafe.park(Unsafe.java:-1)
      at java.util.concurrent.locks.LockSupport.park(LockSupport.java:371)
      at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(AbstractQueuedSynchronizer.java:519)
      at java.util.concurrent.ForkJoinPool.unmanagedBlock(ForkJoinPool.java:3780)
      at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3725)
      at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1707)
      at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
      at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1070)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
      at java.lang.Thread.runWith(Thread.java:1596)
      at java.lang.Thread.run(Thread.java:1583)

"PostgreSQL-JDBC-Cleaner@12465" tid=0x4e nid=NA waiting
  java.lang.Thread.State: WAITING
      at jdk.internal.misc.Unsafe.park(Unsafe.java:-1)
      at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:269)
      at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1847)
      at java.lang.ref.ReferenceQueue.await(ReferenceQueue.java:71)
      at java.lang.ref.ReferenceQueue.remove0(ReferenceQueue.java:143)
      at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:218)
      at org.postgresql.util.LazyCleaner$1.run(LazyCleaner.java:128)
      at java.lang.Thread.runWith(Thread.java:1596)
      at java.lang.Thread.run(Thread.java:1583)

"SpringApplicationShutdownHook@26510" tid=0x4a nid=NA waiting
  java.lang.Thread.State: WAITING
      at jdk.internal.misc.Unsafe.park(Unsafe.java:-1)
      at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:269)
      at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1758)
      at java.util.concurrent.ThreadPoolExecutor.awaitTermination(ThreadPoolExecutor.java:1475)
      at java.util.concurrent.Executors$DelegatedExecutorService.awaitTermination(Executors.java:780)
      at java.util.concurrent.ExecutorService.close(ExecutorService.java:417)
      at org.springframework.beans.factory.support.DisposableBeanAdapter.destroy(DisposableBeanAdapter.java:232)
      at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroyBean(DefaultSingletonBeanRegistry.java:587)
      at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingleton(DefaultSingletonBeanRegistry.java:559)
      at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingleton(DefaultListableBeanFactory.java:1202)
      at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons(DefaultSingletonBeanRegistry.java:520)
      at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingletons(DefaultListableBeanFactory.java:1195)
      at org.springframework.context.support.AbstractApplicationContext.destroyBeans(AbstractApplicationContext.java:1184)
      at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:1145)
      at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.doClose(ServletWebServerApplicationContext.java:174)
      at org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:1091)
      at org.springframework.boot.SpringApplicationShutdownHook.closeAndWait(SpringApplicationShutdownHook.java:145)
      at org.springframework.boot.SpringApplicationShutdownHook$$Lambda/0x000001a7826aa2c8.accept(Unknown Source:-1)
      at java.lang.Iterable.forEach(Iterable.java:75)
      at org.springframework.boot.SpringApplicationShutdownHook.run(SpringApplicationShutdownHook.java:114)
      at java.lang.Thread.runWith(Thread.java:1596)
      at java.lang.Thread.run(Thread.java:1583)

"testcontainers-ryuk@4563" tid=0x32 nid=NA waiting
  java.lang.Thread.State: WAITING
      at java.lang.Object.wait0(Object.java:-1)
      at java.lang.Object.wait(Object.java:366)
      at org.testcontainers.utility.RyukResourceReaper.lambda$null$1(RyukResourceReaper.java:115)
      at org.testcontainers.utility.RyukResourceReaper$$Lambda/0x000001a7812beef0.run(Unknown Source:-1)
      at org.rnorth.ducttape.ratelimits.RateLimiter.doWhenReady(RateLimiter.java:27)
      at org.testcontainers.utility.RyukResourceReaper.lambda$maybeStart$2(RyukResourceReaper.java:101)
      at org.testcontainers.utility.RyukResourceReaper$$Lambda/0x000001a7812becd0.run(Unknown Source:-1)
      at java.lang.Thread.runWith(Thread.java:1596)
      at java.lang.Thread.run(Thread.java:1583)

Answer 1

I had a similar issue with SpringBootTest 3.3.1 hanging when switching to JDK 21.

For me it turned out to be some @Scheduled tasks for Springs @EnableScheduling scheduled task execution capability.

It was hanging during bean destruction while destroying the taskExecutor bean org.springframework.security.concurrent.DelegatingSecurityContextScheduledExecutorService:

Thread [SpringApplicationShutdownHook] (Suspended)  
DelegatingSecurityContextScheduledExecutorService(DelegatingSecurityContextExecutorService).awaitTermination(long, TimeUnit) line: 88   
DelegatingSecurityContextScheduledExecutorService(ExecutorService).close() line: 417    
DisposableBeanAdapter.destroy() line: 232   
DefaultListableBeanFactory(DefaultSingletonBeanRegistry).destroyBean(String, DisposableBean) line: 587  
DefaultListableBeanFactory(DefaultSingletonBeanRegistry).destroySingleton(String) line: 559 
DefaultListableBeanFactory.destroySingleton(String) line: 1202  
DefaultListableBeanFactory(DefaultSingletonBeanRegistry).destroySingletons() line: 520  
DefaultListableBeanFactory.destroySingletons() line: 1195   
GenericWebApplicationContext(AbstractApplicationContext).destroyBeans() line: 1186  
GenericWebApplicationContext(AbstractApplicationContext).doClose() line: 1147   
GenericWebApplicationContext(AbstractApplicationContext).close() line: 1093 
SpringApplicationShutdownHook.closeAndWait(ConfigurableApplicationContext) line: 145    
0x0000000801614900.accept(Object) line: not available   
LinkedHashSet<E>(Iterable<T>).forEach(Consumer<? super T>) line: 75 
SpringApplicationShutdownHook.run() line: 114   
Thread.runWith(Object, Runnable) line: 1596 
Thread.run() line: 1583 

The Java 17 ExecutorService wasn't AutoCloseable and had no close method. The destroy bean just called shutdown and continued.

public interface ExecutorService extends Executor

The Java 19+ ExecutorService is AutoCloseable and close waits for 1 day.

public interface ExecutorService extends Executor, AutoCloseable
...
@Override
default void close() {
    boolean terminated = isTerminated();
    if (!terminated) {
        shutdown();
        boolean interrupted = false;
        while (!terminated) {
            try {
                terminated = awaitTermination(1L, TimeUnit.DAYS);
            } catch (InterruptedException e) {
...

My Scheduled tasks (ScheduledThreadPoolExecutors) were on a cron schedule so it wouldn't shutdown the taskExecutor and took a day to timeout.

I didn't need the Scheduled jobs for integration tests so I was able to just exclude their configuration/creation instead of figuring out how to remove them from the work queue.