Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

calling ExecutorService.shutDown() in java

i am starting to learn the ExecutorService class. The documentation (and tutorials online) say to always call ExecutorService.shutDown() to reclaim resources. however, the documentation also says that after you call shutDown(), no new tasks will be accepted. so, my question is, do i have always have to instantiate a new ExecutorService whenever i need to parallelize data processing?

right now i have a List of Callable objects, and i do the following.

public void someMethod() {
 List<OuterCallable> outerCallables = getOuterCallables();
 ExecutorService executor = Executor.newFixedThreadPool(NUM_CPUS);
 executor.invokeAll(tasks);
 executor.shutDown();
}

however, my OuterCallable also splits data or executes data processing in parallel using InnerCallable.

public class OuterCallable implements Callable<Long> {
 public Long call() throws Exception {
  long result = 0L;

  List<InnerCallable> innerCallables = getInnerCallables();
  ExecutorServices executor = Executor.newFixedThreadPool(NUM_CPUS);
  executor.invokeAll(tasks);
  executor.shutDown();

  return result;
 }
}    

i can't remember if it was for ExecutorService or the Fork/Join approach, but i remember the documentations and tutorials saying that the actual parallel procedure to manipulate data should not involve I/O operations and everything should be done in memory. however, in my InnerCallable, i am actually making JDBC calls (not shown here).

ultimately, the way i am using ExecutorService works, but i still have lingering concerns.

  1. is my approach above good programming practice using ExecutorService?
  2. should i be using a singleton instance of ExecutorService?
  3. should i be not only avoiding I/O operations inside my parallel methods but also JDBC calls as well?

as a last concern, i was trying to research a little bit on Fork/Join vs ExecutorService. i came across an article that completely blasted the Fork/Join API/classes. is it worth it to learn Fork/Join? i saw a few articles on stackoverflow and elsewhere, where tests are used to compare Fork/Join vs ExecutorService, and there are graphs showing better CPU usage of Fork/Join vs ExecutorService (via Windows Task Manager). however, when i use ExecutorService (JDK 1.7.x), my CPU usage is max. has ExecutorService improved with the latest JDK?

any help/guidance is appreciated.

like image 884
Jane Wayne Avatar asked Feb 27 '12 13:02

Jane Wayne


People also ask

How do I shut down ExecutorService?

To properly shut down an ExecutorService, we have the shutdown() and shutdownNow() APIs. The shutdown() method doesn't cause immediate destruction of the ExecutorService. It will make the ExecutorService stop accepting new tasks and shut down after all running threads finish their current work: executorService.

Does ExecutorService shutdown automatically?

Using shutdown() and awaitTermination​() In general, the ExecutorService will not be automatically destroyed when there is no task to process. It will stay alive and wait for new tasks to come.

Do we need to close ExecutorService in Java?

An unused ExecutorService should be shut down to allow reclamation of its resources. Method submit extends base method Executor.


1 Answers

You should add awaitTermination calls, because shutDown returns without waiting for Callables to finish. Other than that,

  1. Do you OuterCallables have sequential dependencies? If so, your approach is fine, but using ForkJoinPool would be preferable because it will keep the number of worker threads low. If not, it would be better to submit a big flattened collection of Callables to a single Executor.
  2. Only if you want to use the ExecutorService in several different routines and want to avoid passing it around. If it's only used in someMethod, might as well instantiate it there as you are doing.
  3. You should avoid I/O routines that will take a long time to complete. If your ExecutorService has 4 worker threads and all of them are blocking on I/O, the JVM won't be using the CPU at all, even though other Callables may be waiting to do CPU-intensive work. A few JDBC calls is okay as long as the queries do not take long to complete.
like image 195
Daniel Lubarov Avatar answered Sep 20 '22 15:09

Daniel Lubarov