Is a variable swap guaranteed to be atomic in python?

Question

With reference to the following link: http://docs.python.org/faq/library.html#what-kinds-of-global-value-mutation-are-thread-safe

I wanted to know if the following:

(x, y) = (y, x) 

will be guaranteed atomic in cPython. (x and y are both python variables)

Answer 1

Let's see:

>>> x = 1 >>> y = 2 >>> def swap_xy(): ...   global x, y ...   (x, y) = (y, x) ...  >>> dis.dis(swap_xy)   3           0 LOAD_GLOBAL              0 (y)               3 LOAD_GLOBAL              1 (x)               6 ROT_TWO                            7 STORE_GLOBAL             1 (x)              10 STORE_GLOBAL             0 (y)              13 LOAD_CONST               0 (None)              16 RETURN_VALUE     

It doesn't appear that they're atomic: the values of x and y could be changed by another thread between the LOAD_GLOBAL bytecodes, before or after the ROT_TWO, and between the STORE_GLOBAL bytecodes.

If you want to swap two variables atomically, you'll need a lock or a mutex.

For those desiring empirical proof:

>>> def swap_xy_repeatedly(): ...   while 1: ...     swap_xy() ...     if x == y: ...       # If all swaps are atomic, there will never be a time when x == y. ...       # (of course, this depends on "if x == y" being atomic, which it isn't; ...       #  but if "if x == y" isn't atomic, what hope have we for the more complex ...       #  "x, y = y, x"?) ...       print 'non-atomic swap detected' ...       break ...  >>> t1 = threading.Thread(target=swap_xy_repeatedly) >>> t2 = threading.Thread(target=swap_xy_repeatedly) >>> t1.start() >>> t2.start() >>> non-atomic swap detected 

Answer 2

Yes, yes it will.

I stand corrected.

Kragen Sitaker writes:

Someone recommended using the idiom

spam, eggs = eggs, spam 

to get a thread-safe swap. Does this really work? (...)
So if this thread loses control anywhere between the first LOAD_FAST
and the last STORE_FAST, a value could get stored by another thread
into "b" which would then be lost. There isn't anything keeping this
from happening, is there?

Nope. In general not even a simple assignment is necessarily thread safe since performing the assignment may invoke special methods on an object which themselves may require a number of operations. Hopefully the object will have internally locked its "state" values, but that's not always the case.

But it's really dictated by what "thread safety" means in a particular application, because to my mind there are many levels of granularity of such safety so it's hard to talk about "thread safety". About the only thing the Python interpreter is going to give you for free is that a built-in data type should be safe from internal corruption even with native threading. In other words if two threads have a=0xff and a=0xff00, a will end up with one or the other, but not accidentally 0xffff as might be possible in some other languages if a isn't protected.

With that said, Python also tends to execute in such a fashion that you can get away with an awful lot without formal locking, if you're willing to live on the edge a bit and have implied dependencies on the actual objects in use. There was a decent discussion along those lines here in c.l.p a while back - search groups.google.com for the "Critical sections and mutexes" thread among others.

Personally, I explicitly lock shared state (or use constructs designed for exchanging shared information properly amongst threads, such as Queue.Queue) in any multi-threaded application. To my mind it's the best protection against maintenance and evolution down the road.

-- -- David