Python's multiprocessing Does Not Play Nicely With threading.local?

Question

I have two processes (see sample code) that each attempt to access a threading.local object. I would expect the below code to print "a" and "b" (in either order). Instead, I get "a" and "a". How can I elegantly and robustly reset the threading.local object when I startup whole new processes?

import threading
import multiprocessing
l = threading.local()
l.x = 'a'
def f():
    print getattr(l, 'x', 'b')
multiprocessing.Process(target=f).start()
f()

edit: For reference, when I use threading.Thread instead of multiprocessing.Process, it works as expected.

Answer 1

There is now a multiprocessing-utils (github) library on pypi with a multiprocessing-safe version of threading.local() which can be pip installed.

It works by wrapping a standard threading.local() and checking that the PID has not changed since it was last used (as per the answer here from @immortal).

Use it exactly like threading.local():

l = multiprocessing_utils.local()
l.x = 'a'
def f():
    print getattr(l, 'x', 'b')
f()                                        # prints "a"
threading.Thread(target=f).start()         # prints "b"
multiprocessing.Process(target=f).start()  # prints "b"

Full disclosure: I just created this module