Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Java parallel work iterator?

I'm looking for a class where I can override a method to do the work, and return the results like an iterator. Something like this:

ParallelWorkIterator<Result> itr = new ParallelWorkIterator<Result>(trials,threads) {

  public Result work() {
    //do work here for a single trial...
    return answer;
  }

};
while (itr.hasNext()) {
  Result result = itr.next();
  //process result...
}

This is mainly going to be used for things like monte carlo simulations, but I don't want to have to deal with setting up thread pools and managing returning threads every time. I rolled my own class that hopefully accomplishes this, but I'm not confident enough in it and thought I'd check if something like this already existed.

Edit: To be clear, I want it to keep running in the background and queuing results after each work method returns until all trials have been completed. So the next method may wait to return until there is a result in the queue.

like image 818
job Avatar asked Aug 04 '09 16:08

job


People also ask

What is difference between iterator and Spliterator?

Like Iterator and ListIterator, Spliterator is a Java Iterator, which is used to iterate elements one-by-one from a List implemented object. The main functionalities of Spliterator are: Splitting the source data. Processing the source data.

Do Java threads run in parallel?

Java Thread allows us to create a lightweight process that executes some tasks. We can create multiple threads in our program and start them. Java runtime will take care of creating machine-level instructions and work with OS to execute them in parallel.

Is parallel stream multithreading?

A parallel stream is executed by different threads, running on multiple CPU cores in a computer. The stream elements are split into substreams that are processed by multiple instances of the stream pipeline being executed in multiple threads.


1 Answers

Have a look at the ExecutorCompletionService. It does everything you want.

   void solve(Executor e, Collection<Callable<Result>> solvers)
     throws InterruptedException, ExecutionException {
       //This class will hold and execute your tasks
       CompletionService<Result> ecs
           = new ExecutorCompletionService<Result>(e);
       //Submit (start) all the tasks asynchronously
       for (Callable<Result> s : solvers)
           ecs.submit(s);
       //Retrieve completed task results and use them
       int n = solvers.size();
       for (int i = 0; i < n; ++i) {
           Result r = ecs.take().get();
           if (r != null)
               use(r);
       }
   }

The benefit of using a CompletionService is that it always returns the first completed result. This ensures you're not waiting for tasks to complete and it lets the uncompleted tasks run in the background.

like image 198
Tim Avatar answered Sep 22 '22 12:09

Tim