Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Java equivalent for Python pool.map/ Multiprocessing

I was wondering if somebody could point me to a simple equivalent of python's multiprocessing module in java.

I have a simple parallel processing scenario (where no 2 processes interact): Take a data set and split it into 12 and apply a java method to the 12 datasets, collect results and join them in a list of some sort with the same ordering.

Java being a "pro" language appears to have multiple libraries and methods - anyone who can help this java newbie get started?

I would like to do this with minimal of coding - as i said my requirement is pretty straightforward.

Update: how to do multiprocessing in java, and what speed gains to expect?

This seems to indicate threads is the way to go. I expect I have no choice but wade into a bunch of locks (pun unintended) and wait for my ship to sail. Simple examples are welcome nevertheless.

like image 361
pythOnometrist Avatar asked Oct 04 '22 05:10

pythOnometrist


1 Answers

There's no exactly-compatible class, but ExecutorService gives you everything you need to implement it.

In particular, there's no function to map a Callable over a Collection and wait on the results, but you can easily build a Collection<Callable<T>> out of a Callable<T> and Collection<T>, then just call invokeAll, which returns you a List<Future<T>>.

(If you want to emulate some of the other functions from multiprocessing.Pool, you will need to loop around submit instead and build your own collection of things to wait on. But map is simple.)

like image 94
abarnert Avatar answered Oct 11 '22 13:10

abarnert