I am trying to speed up a program that uses ExecutorService
to run tasks in parallel. It basically works like this:
Initialize a fixed size thread pool of size n
Read a bunch (about 2500 files) of XML files containing input for the tasks
Process the XML files using the worker threads from the pool
Everything works as expected, but the problem I have is that each worker thread has an instance of a class that does calculations on the input data. This instance is stored in a ThreadLocal
. Now, all these thread local instances are created when the correspondig worker thread is started, that means after all XML input files are read.
Since initialization of the calculation objects takes quite some time, I'd rather have the thread pool initialize all worker threads right from the start, so that the initializations of the calculation objects can run parallel to the reading of the input files.
Here is some code to help you get an idea about how it currently works (I stripped code not related to the problem).
Initializing the thread pool and the thread local:
private final ExecutorService executor = Executors.newFixedThreadPool(Math.max(1, Runtime
.getRuntime().availableProcessors() - 1));
private ThreadLocal<Calculator> calculator = new ThreadLocal<Calculator>() {
@Override
protected Calculator initialValue() {
try {
Calculator instance = createCalculator();
return instance;
} catch (Throwable e) {
throw new RuntimeException(e);
}
}
};
Submitting a new calculation:
@Override
public FutureTask<Output> calc(Input input) {
FutureTask<Output> task = new FutureTask<>(
new Callable<Rueckgabe>() {
@Override
public Output call() throws Exception {
try {
return calculator.get().calc(input);
} catch (Throwable e) {
System.err.println("Exception: " + e.getMessage());
e.printStackTrace(System.err);
return null;
}
}
});
executor.execute(task);
return task;
}
What is the right way to have the ExecutorService
start all worker threads for the pool right from the start? Or do I have to something like n
dummy tasks to force initialization?
PS: I have to use Java 7 for the foreseeable future due to IT restrictions.
To use thread pools, we first create a object of ExecutorService and pass a set of tasks to it. ThreadPoolExecutor class allows to set the core and maximum pool size. The runnables that are run by a particular thread are executed sequentially.
When using an Executor, we can shut it down by calling the shutdown() or shutdownNow() methods. Although, it won't wait until all threads stop executing. Waiting for existing threads to complete their execution can be achieved by using the awaitTermination() method.
We can use join() methodto ensure all threads that started from main must end in order in which they started and also main should end in last.In other words waits for this thread to die. Calling join() method internally calls join(0);
Remember: If corePoolSize or more threads are running, the executor prefers queuing the task than creating a new thread. If the queue is full and creating a new thread would exceed maximumPoolSize the submitted task is rejected by the executor.
Add a Threadfactory to the Executor which would do the initialization.
executor = Executors.newFixedThreadPool(numprocrssors, new ThreadFactory ( ){
public Thread newThread(Runnable r) {
return new Thread(r){
{calculator.get();} // this is an initialization statement, added to all constructors.
};
}});
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With