Menu
Whats a simple code that does parallel processing in python 2.7? All the examples Ive found online are convoluted and include unnecessary codes.
how would i do a simple brute force integer factoring program where I can factor 1 integer on each core (4)? my real program probably only needs 2 cores, and need to share information.
I know that parallel-python and other libraries exist, but i want to keep the number of libraries used to a minimum, thus I want to use the thread and/or multiprocessing libraries, since they come with python
627
You can use joblib library to do parallel computation and multiprocessing. You can find a complete explanation of the python and R multiprocessing with couple of examples here. Show activity on this post.
In parallel processing, we take in multiple different forms of information at the same time. This is especially important in vision. For example, when you see a bus coming towards you, you see its color, shape, depth, and motion all at once. If you had to assess those things one at a time, it would take far too long.
NumPy does not run in parallel. On the other hand Numba fully utilizes the parallel execution capabilities of your computer. NumPy functions are not going to use multiple CPU cores, never mind the GPU.
A good simple way to start with parallel processing in python is just the pool mapping in mutiprocessing -- its like the usual python maps but individual function calls are spread out over the different number of processes.
Factoring is a nice example of this - you can brute-force check all the divisions spreading out over all available tasks:
from multiprocessing import Pool import numpy numToFactor = 976 def isFactor(x): result = None div = (numToFactor / x) if div*x == numToFactor: result = (x,div) return result if __name__ == '__main__': pool = Pool(processes=4) possibleFactors = range(1,int(numpy.floor(numpy.sqrt(numToFactor)))+1) print 'Checking ', possibleFactors result = pool.map(isFactor, possibleFactors) cleaned = [x for x in result if not x is None] print 'Factors are', cleaned This gives me
Checking [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31] Factors are [(1, 976), (2, 488), (4, 244), (8, 122), (16, 61)] mincemeat is the simplest map/reduce implementation that I've found. Also, it's very light on dependencies - it's a single file and does everything with standard library.
43
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
