Efficient Number of Threads

Question

I want to optimize my application number of threads. Almost all of them have IO beside CPU usage in an equal value. How much is the efficient number of threads when there are no other applications running in system. I want the answer for Windows and under JVM.

Answer 1

There is no per-OS answer. This will depend on the specific set of tasks that your code is performing. You should benchmark your application with different configurations to see which one is the most performant.

Some general tips about multithreading:

• You can't speed up like tasks with more threads; the exception is that if you have multiple CPUs, you can parallelize compute tasks with one thread per CPU, provided this logic can be split up such that it does not necessarily need to be executed serially. A good example for this would be a divide-and-conquer problem like mergesort, where the two halves can be sorted in any order.

• You can achieve some speedup by parallelizing tasks that do not make use of the same part of the machine. So, given that you say you have "equal value" of I/O and compute tasks, you will want to separate those into different threads - again, this assumes that ordering is not important.

If it is the case (as with many applications) that threads perform some compute logic followed by some I/O (like writing the data to disk or a database server, for example) then it will be very difficult to come up with some formula to determine the exact number of threads you should have, as this will be highly dependent on the data you are processing, how you are processing it, and what you are doing with it when processing is done. This is why the best thing to do is have a configurable thread pool whose size can be adjusted easily - then run some load tests with different sizes and see which one performed best.