Thread in Python : class attribute (list) not thread-safe?

Question

I am trying to understand Threads in Python.

The code

And now I have a problem, which I have surrounded in one simple class:

# -*- coding: utf-8 -*-
import threading

class myClassWithThread(threading.Thread):

    __propertyThatShouldNotBeShared = []
    __id = None
    def __init__(self, id):
        threading.Thread.__init__(self)
        self.__id = id

    def run(self):
        while 1:
            self.dummy1()
            self.dummy2()

    def dummy1(self):
        if self.__id == 2:
            self.__propertyThatShouldNotBeShared.append("Test value")


    def dummy2(self):
        for data in self.__propertyThatShouldNotBeShared:
            print self.__id
            print data
            self.__propertyThatShouldNotBeShared.remove(data)



obj1 = myClassWithThread(1)
obj2 = myClassWithThread(2)
obj3 = myClassWithThread(3)

obj1.start()
obj2.start()
obj3.start()

Description

Here is what the class does : The class has two attributes :

• __id which is an identifier for the object, given when the constructor is called
• __propertyThatShouldNotBeShared is a list and will contain a text value

Now the methods

• run() contains an infinite loop in which I call dummy1() and then dummy2()
• dummy1() which adds to attribute (list) __propertyThatShouldNotBeShared the value "Test value" only IF the __id of the object is equal to 2
• dummy2() checks if the size of the list __propertyThatShouldNotBeShared is strictly superior to 0, then
  • for each value in __propertyThatShouldNotBeShared it prints the id of
  • the object and the value contained in __propertyThatShouldNotBeShared
  • then it removes the value

Here is the output that I get when I launch the program :

21

Test valueTest value

2
Test value
Exception in thread Thread-2:
Traceback (most recent call last):
  File "E:\PROG\myFace\python\lib\threading.py", line 808, in __bootstrap_inner
    self.run()
  File "E:\PROG\myFace\myProject\ghos2\src\Tests\threadDeMerde.py", line 15, in run
    self.dummy2()
  File "E:\PROG\myFace\myProject\ghos2\src\Tests\threadDeMerde.py", line 27, in dummy2
    self.__propertyThatShouldNotBeShared.remove(data)
ValueError: list.remove(x): x not in list

The problem

As you can see in the first line of the output I get this "1"...which means that, at some point, the object with the id "1" tries to print something on the screen...and actually it does! But this should be impossible! Only object with id "2" should be able to print anything!

What is the problem in this code ? Or what is the problem with my logic?

Answer 1

The problem is this:

class myClassWithThread(threading.Thread):
    __propertyThatShouldNotBeShared = []

It defines one list for all objects which is shared. You should do this:

class myClassWithThread(threading.Thread):
    def __init__(self, id):
        self.__propertyThatShouldNotBeShared = []
        # the other code goes here

Answer 2

There are two problems here—the one you asked about, thread-safety, and the one you didn't, the difference between class and instance attributes.

---

It's the latter that's causing your actual problem. A class attribute is shared by all instances of the class. It has nothing to do with whether those instances are accessed on a single thread or on multiple threads; there's only one __propertyThatShouldNotBeShared that's shared by everyone. If you want an instance attribute, you have to define it on the instance, not on the class. Like this:

class myClassWithThread(threading.Thread):
    def __init__(self, id):
        self.__propertyThatShouldNotBeShared = []

---

Once you do that, each instance has its own copy of __propertyThatShouldNotBeShared, and each lives on its own thread, so there is no thread-safety issue possible.

However, your original code does have a thread-safety problem.

Almost nothing is automatically thread-safe (aka "synchronized"); exceptions (like queue.Queue) will say so explicitly, and be meant specifically for threaded programming.

You can avoid this in three ways:

• Don't share anything.
• Don't mutate anything you share.
• Don't mutate anything you share unless it's protected by an appropriate synchronization object.

The last one is of course the most flexible, but also the most complicated. In fact, it's at the center of why people consider threaded programming hard.

The short version is, everywhere you modify or access shared mutable data like self.__propertyThatShouldNotBeShared, you need to be holding some kind of synchronization object, like a Lock. For example:

class myClassWithThread(threading.Thread):
    __lock = threading.Lock()
    # etc.

    def dummy1(self):
        if self.__id == 2:
            with self.__lock:
                self.__propertyThatShouldNotBeShared.append("Test value")

If you stick to CPython, and to built-in types, you can often get away with ignoring locks. But "often" in threaded programming is just a synonym for "always during testing and debugging, right up until the release or big presentation, when it suddenly begins failing". Unless you want to learn the rules for how the Global Interpreter Lock and the built-in types work in CPython, don't rely on this.