Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to initialize all threads of a fixed thread pool before submitting any tasks? (JAVA)

I am trying to speed up a program that uses ExecutorService to run tasks in parallel. It basically works like this:

  1. Initialize a fixed size thread pool of size n

  2. Read a bunch (about 2500 files) of XML files containing input for the tasks

  3. Process the XML files using the worker threads from the pool

Everything works as expected, but the problem I have is that each worker thread has an instance of a class that does calculations on the input data. This instance is stored in a ThreadLocal. Now, all these thread local instances are created when the correspondig worker thread is started, that means after all XML input files are read.

Since initialization of the calculation objects takes quite some time, I'd rather have the thread pool initialize all worker threads right from the start, so that the initializations of the calculation objects can run parallel to the reading of the input files.

Here is some code to help you get an idea about how it currently works (I stripped code not related to the problem).

Initializing the thread pool and the thread local:

  private final ExecutorService executor = Executors.newFixedThreadPool(Math.max(1, Runtime
      .getRuntime().availableProcessors() - 1));

  private ThreadLocal<Calculator> calculator = new ThreadLocal<Calculator>() {
    @Override
    protected Calculator initialValue() {
      try {
        Calculator instance = createCalculator();
        return instance;
      } catch (Throwable e) {
        throw new RuntimeException(e);
      }
    }
  };

Submitting a new calculation:

  @Override
  public FutureTask<Output> calc(Input input) {
    FutureTask<Output> task = new FutureTask<>(
        new Callable<Rueckgabe>() {
          @Override
          public Output call() throws Exception {
            try {
              return calculator.get().calc(input);
            } catch (Throwable e) {
              System.err.println("Exception: " + e.getMessage());
              e.printStackTrace(System.err);
              return null;
            }
          }
        });
    executor.execute(task);
    return task;
  }

What is the right way to have the ExecutorService start all worker threads for the pool right from the start? Or do I have to something like n dummy tasks to force initialization?

PS: I have to use Java 7 for the foreseeable future due to IT restrictions.

like image 715
Axel Avatar asked Aug 07 '15 08:08

Axel


People also ask

How do you initialize a thread pool in Java?

To use thread pools, we first create a object of ExecutorService and pass a set of tasks to it. ThreadPoolExecutor class allows to set the core and maximum pool size. The runnables that are run by a particular thread are executed sequentially.

Which method waits till all threads initiated by the service () method have finished?

When using an Executor, we can shut it down by calling the shutdown() or shutdownNow() methods. Although, it won't wait until all threads stop executing. Waiting for existing threads to complete their execution can be achieved by using the awaitTermination() method.

How can you ensure all threads that started from Main?

We can use join() methodto ensure all threads that started from main must end in order in which they started and also main should end in last.In other words waits for this thread to die. Calling join() method internally calls join(0);

What happens if you submit a task when the queue of the thread pool is already filled?

Remember: If corePoolSize or more threads are running, the executor prefers queuing the task than creating a new thread. If the queue is full and creating a new thread would exceed maximumPoolSize the submitted task is rejected by the executor.


1 Answers

Add a Threadfactory to the Executor which would do the initialization.

executor = Executors.newFixedThreadPool(numprocrssors, new ThreadFactory ( ){
    public Thread newThread(Runnable r) {
        return new Thread(r){
            {calculator.get();} // this is an initialization statement, added to all constructors.
        };
    }});
like image 123
Alexei Kaigorodov Avatar answered Nov 01 '22 11:11

Alexei Kaigorodov