Parallelizing multiplication of vectors-like computation in python

Question

I have got a chunk of code like

for i in range(0, len(a))
    b[i] = func(a[i])

where a and b are arrays of the same length, a is given (and big), func is some function that has a lot of local variables but does not use any global variables.

I would like to distribute computations of func across several CPUs. Presumably I need to use multiprocessing module, but I have not found any relevant examples. Could you help? Thanks.

Answer 1

See the very first code example in the multiprocessing docs:

from multiprocessing import Pool

# you could define `func`, `a` here

if __name__=="__main__":
    p = Pool() # use all available CPU cores
    b = p.map(func, a)

Answer 2

Use process pool. You can see a full sample in my github: https://github.com/mariazverina/codejam/blob/master/src/main.py

from multiprocessing import Pool

p = Pool(4)  # set to number of cores
b = p.map(func, a)