How many cpu cores can a single java process make use of?

Question

I'm running some performance tests to determine response times and ability to handle concurrency with various cpu / ram / os configurations. One interesting find I've run into, is that, it looks like a single jvm performs better with 4 cores than it does with 2 cores (no surprise there), but adding more cores beyond the 4th core does not result in any significant improvements. But then, adding another jvm instance with a load balancer (same hardware) resulted in a dramatic improvement.

It looks like an individual process is limited in the number of cores it can utilize, perhaps due to a limitation on the number of os threads a process can spawn at a time. This is a 64 bit environment.

Im using tomcat and tried changing the "maxThreads" attribute but that didn't make a difference for the amount of concurrency Im looking to handle.

Any other reason this could be explained?

Answer 1

In general, a Java application will try to schedule every active thread on a separate core. This includes the GC threads. If the application was not CPU-bound in the first place, then adding more cores would not make any difference as the threads are blocked by something.

For a web application, things get a little more complicated. Increasing the size of the thread pool won't make any difference if there is something else that the threads are waiting on, like a database from a database connection pool, for example. You need to look carefully at your load balancer and Tomcat configurations. If those are not tuned properly it is very easy to have the behavior you describe.

I wouldn't worry too much about the JVM being limited in the number of threads it can have. I routinely see over 600 threads in every JVM in our production system. We use 8 core machines and the requests are short in duration, so most of these threads are in an I/O wait state.