How to optimize number of threads to speed up processing

Question

I have read many similar questions . However I was not quite satisfied with answers.

I would like to build an algorithm that would adjust the number of threads depending on the average speed.

Let's say as I introduce a new thread, the average speed of task execution increases , it means that the new thread is good. Then the algorithm should try to add another thread ... until the optimal number of threads is achieved .......

Also the algorithm should be keeping track of the average speed. If at some point the average speed goes down significantly, let's say by 10 % (for any reason e.g. i open a different application or whatever) , then the algorithm should terminate one thread and see if the speed goes up ...

Maybe such an API exists. Please, give me any directions or any code example how I could implement such an algorithm

Thank You !

Answer 1

I do not know self-tune system that you are describing but it sounds like not so complicated task once you are using ready thread pool. Take thread pool from concurrency package, implement class TimeConsumptionCallable implements Callable that wraps any other callable and just measures the execution time.

Now you just have to change (increase or decrease) number of working threads when average execution time increases or decreases.

Just do not forget that you need enough statistics before you decide to change number of working threads. Otherwise various random effects that do not depend on your application can cause your thread pool to grow and go down all the time that can itself kill overall performance.

Answer 2

newCachedThreadPool() V/s newFixedThreadPool suggests that perhaps you should be looking at ExecutorService.newCachedThreadPool().

Creates a thread pool that creates new threads as needed, but will reuse previously constructed threads when they are available. These pools will typically improve the performance of programs that execute many short-lived asynchronous tasks. Calls to execute will reuse previously constructed threads if available. If no existing thread is available, a new thread will be created and added to the pool. Threads that have not been used for sixty seconds are terminated and removed from the cache. Thus, a pool that remains idle for long enough will not consume any resources. Note that pools with similar properties but different details (for example, timeout parameters) may be created using ThreadPoolExecutor constructors.