Java threading optimization at 100% CPU usage

Question

I have an application that accepts work on a queue and then spins that work off to be completed on independent threads. The number of threads is not massive, say up to 100, but these are intensive tasks and can quickly bump the CPU up to 100%.

To get the most work done the quickest: am I best off to just launch more threads when I need to do more work and let the Java thread scheduler handle distributing the work, or would getting smarter and managing the work load to keep the CPU below 100% get me further faster?

The machine is dedicated to my java app.

EDIT:

Thanks for the fantastic input!

The tasks are of varying complexity and involve I/O so having a low thread pool of say 4 may well be only running the CPU to 20%. I have no way of knowing how many tasks will actually take the CPU to 100%.

My thought was should I monitor CPU through RMI and dynamically dial the work up and down, or should I just not care and let the OS handle it.

Answer 1

If you have too many simultaneous compute-intensive tasks in parallel threads, you reach the point of diminishing returns very quickly. In fact, if there are N processors (cores), then you don't want more than N such threads. Now, if the tasks occasionally pause for I/O or user interaction, then the right number can be somewhat larger. But in general, if at any one moment there are more threads that want to do computation than there are cores available, then your program is wasting time on context switches -- i.e., the scheduling is costing you.