Are locks unnecessary in multi-threaded Python code because of the GIL?

Question

If you are relying on an implementation of Python that has a Global Interpreter Lock (i.e. CPython) and writing multithreaded code, do you really need locks at all?

If the GIL doesn't allow multiple instructions to be executed in parallel, wouldn't shared data be unnecessary to protect?

sorry if this is a dumb question, but it is something I have always wondered about Python on multi-processor/core machines.

same thing would apply to any other language implementation that has a GIL.

Answer 1

No - the GIL just protects python internals from multiple threads altering their state. This is a very low-level of locking, sufficient only to keep python's own structures in a consistent state. It doesn't cover the application level locking you'll need to do to cover thread safety in your own code.

The essence of locking is to ensure that a particular block of code is only executed by one thread. The GIL enforces this for blocks the size of a single bytecode, but usually you want the lock to span a larger block of code than this.

Answer 2

You will still need locks if you share state between threads. The GIL only protects the interpreter internally. You can still have inconsistent updates in your own code.

For example:

#!/usr/bin/env python import threading  shared_balance = 0  class Deposit(threading.Thread):     def run(self):         for _ in xrange(1000000):             global shared_balance             balance = shared_balance             balance += 100             shared_balance = balance  class Withdraw(threading.Thread):     def run(self):         for _ in xrange(1000000):             global shared_balance             balance = shared_balance             balance -= 100             shared_balance = balance  threads = [Deposit(), Withdraw()]  for thread in threads:     thread.start()  for thread in threads:     thread.join()  print shared_balance 

Here, your code can be interrupted between reading the shared state (balance = shared_balance) and writing the changed result back (shared_balance = balance), causing a lost update. The result is a random value for the shared state.

To make the updates consistent, run methods would need to lock the shared state around the read-modify-write sections (inside the loops) or have some way to detect when the shared state had changed since it was read.