Problems with pool size of Spring's ThreadPoolTaskExecutor

Question

I'm doing some load tests agains my Spring application and now I'm a little bit confused about the configuration of the ThreadPoolTaskExecutor.

The documentation of the internally used ThreadPoolExecutor describes the corePoolSize as "the number of threads to keep in the pool, even if they are idle, [...]" and maximumPoolSize as "the maximum number of threads to allow in the pool".

That obviously means that the maximumPoolSize limits the number of thread in the pool. But instead the limit seems the be set by the corePoolSize. Actually I configured just the corePoolSize with 100 an let the maximumPoolSize unconfigured (that means the default value is used: Integer.MAX_VALUE = 2147483647).

When I run the load test I can see (by reviewing the logs), that the executed worker thread are numbered from worker-1 to worker-100. So in this case the thread pool size is limited by corePoolSize. Even if I set maximumPoolSize to 200 or 300, the result is exactly the same.

Why the value of maximumPoolSize has no affect in my case?

@Bean
public TaskExecutor taskExecutor() {
    ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
    taskExecutor.setCorePoolSize(100);
    taskExecutor.setThreadNamePrefix("worker-");
    return taskExecutor;
}

SOLUTION

I've found the solution in the documentation: "If there are more than corePoolSize but less than maximumPoolSize threads running, a new thread will be created only if the queue is full". The default queue size is Integer.MAX_VALUE. If I limit the queue, everything works fine.

Answer 1

I have done some testing on ThreadPoolTaskExecutor and there is three things that you have to understand:

• corePoolSize
• queueCapacity
• maxPoolSize

When you start the process there is no threads in the pool. Each time a task comes one new executor thread will be created to handle this new load as long the corePoolSize is not reached. When the corePoolSize is reached the next task will be shift to the queue and wait for a free executor thread. If the load is too high and queueCapacity is full, the new executor threads will be created unless the maxPoolSize is reached. These additional threads will expire as soon as the queue is empty. If the corePoolSize is exhausted, queueCapacity is full and maxPoolSize is also reached then the new submitteds tasks will be rejected and called will get an exception.

You have not mentioned the queueCapacity of your configuration so it might be set to highest integer number and thus maxPoolSize is never getting triggered. Try with small corePoolSize and queueCapacity and you will observe the desired result.

Answer 2

If you have 100 threads in a pool and you are executing CPU bound code on 4 physical CPU cores, most of your core threads are idle in the pool waiting to be re-used. That is probably why you don't see more than worker-100.

You didn't show us code you are executing in workers, therefore I assume it is not I/O bound. If it would be I/O bound code and 100 of your core threads would be occupied by waiting for blocking I/O operations to finish, ThreadPoolExecutor would need to create additional workers.

Try it with corePoolSize lower than number of cores on your machine to confirm. Another option is to put Thread.sleep(1000) into your worker code and observe how your workers count will be raising.

EDIT:

You suggested to use SimpleAsyncTaskExecutor in comment. Notice this section of Spring Framework docs:

SimpleAsyncTaskExecutor This implementation does not reuse any threads, rather it starts up a new thread for each invocation. However, it does support a concurrency limit which will block any invocations that are over the limit until a slot has been freed up. If you are looking for true pooling, see the discussions of SimpleThreadPoolTaskExecutor and ThreadPoolTaskExecutor below.

So with SimpleAsyncTaskExecutor you don't have pooling at all and a lot of resources (CPU cycles included) are wasted on creation and deletion of Thread objects, which may be quite expensive operation.

So SimpleAsyncTaskExecutor executor type does more harm than good to your load testing. If you want to have more workers, use more machines. It's naive to use only one machine if you want to have accurate load testing.