Is python numpy array operation += thread safe?

Question

How can I be sure this code is thread safe ?

import numpy as np
from threading import Thread

n_threads = 5

ones = np.ones((5, 5))
A = np.ones((5, 5))

def my_function():
    global A
    for i in range(250):
        A += ones # is += thread safe ?

threads = [Thread(target=my_function) for i in range(n_threads)]

for t in threads:
    t.start()

for t in threads:
    t.join()

print(A)

Should A be a critical shared memory ? Surprisingly I always get the same result, and all entries of the array all have the same value. I would expect for the threads to update the values of the matrix, and for some to be lost...

Thanks.

Answer 1

Your code is not safe. Some NumPy ufuncs (like add() which you implicitly use) can release the GIL. Probably the reason you never see any problems in your toy example is that it runs for such a short time and the data size is very small. You may also be avoiding problems by the simple nature of your calculation, but I imagine your real code is more complex.

In short, you can't do what you're doing in multiple threads without locking. And locking will probably defeat the purpose of multiple threads for code like this.