Reusing ThreadPoolExecutor vs Creating and Disposing Ad Hoc?

Question

I am building a multithreaded process that has a couple stages, each stage iterating through an unknown number of objects (hundreds of thousands from a buffered query resultset or text file). Each stage will kick off a runnable or callable for each object, but all runnables/callables must complete before moving on to the next stage.

I do not want to use a latch or any kind of synchronizer because I don't want to hurt the throughput. I suspect the latch's internals will slow things down with the synchronized counter. I also don't want to use a list of futures with invokeAll() either because I want to start execution of runnables immediately as I iterate through them.

However, creating a ThreadPoolExecutor for each stage, looping through and submitting all the runnables, and then shutting it down for each stage seems to be a functional solution...

public void runProcess() {

ResultSet rs = someDbConnection.executeQuery(someSQL);

ExecutorService stage1Executor = Executors.newFixedThreadPool(9);
while (rs.next()) { 
//SUBMIT UNKNOWN # OF RUNNABLES FOR STAGE 1
}
rs.close();
stage1Executor.shutdown(); 

rs = someDbConnection.executeQuery(moreSQL);

ExecutorService stage2Executor = Executors.newFixedThreadPool(9);
while (rs.next()) {  
//SUBMIT UNKNOWN # OF RUNNABLES FOR STAGE 2
}
rs.close();
stage2Executor.shutdown();

}

However, I know that setting up threads, threadpools, and anything that involves concurrency is expensive to construct and destroy. Or maybe it is not that big of a deal and I'm just being overly cautious about performance, because concurrency has expensive overhead no matter what. Is there a more efficient way of doing this? Using some kind of wait-for-completion operation I don't know about?

Answer 1

If you destroy the thread-pool and re-init a new one it will likely cost you much more than using a CountDownLatch!

Further, calling stage1Executor.shutdown(); does not promise that all the current threads will finish their execution before the new ExecutorService is up and running. Even calling shutdownNow() cannot guarantee it! (and you probably wouldn't want to call shutdownNow() because you want your threads to finish executing).

Donald Knuth's once said:

premature optimization is the root of all evil.

so even if you are not persuaded by me - better listen to him :)

Answer 2

Setting up and tearing down a handful of thread pools is negligible. Try it out in a loop in a test.

Using a countdown latch is fine, but maybe that might just be duplicating the work that ThreadPoolExecutor does internally and couples your task to your execution framework. Not a fan of this approach.

As for the original code, ExecutorService has an awaitTermination method so you can wait until your work is done before moving to the next stage.

For my money, your pseudo code is fine. Just replace executor.shutdown() with shutdownAndAwaitTermination(ExecutorService), the source for that is here: http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ExecutorService.html