Python: accessing a function by multiple thread concurrently without Lock mechansim

Question

When multiple threads access the same function then do we require to implement the lock mechanism explicitly or not.

I have a program using thread. There are two thread, t1 and t2. t1 is for add1() and t2 is for subtract1().Both of the threads concurrently access the same function myfunction(caller,num)

1. I have defined a simple lock mechanism in the given program using a variable functionLock. Is this reliable or do we need to modify it.

import time, threading

functionLock = '' # blank means lock is open        

def myfunction(caller,num):
    global functionLock
    while functionLock!='': # check and wait until the lock is open
        print "locked by "+ str(functionLock)
        time.sleep(1)

    functionLock = caller # apply lock

    total=0
    if caller=='add1':
        total+=num
        print"1. addition finish with Total:"+str(total)
        time.sleep(2)
        total+=num
        print"2. addition finish with Total:"+str(total)
        time.sleep(2)
        total+=num
        print"3. addition finish with Total:"+str(total)

    else:
        time.sleep(1)
        total-=num
        print"\nSubtraction finish with Total:"+str(total)

    print '\n For '+caller+'() Total: '+str(total)

    functionLock='' # release the lock


def add1(arg1, arg2):

    print '\n START add'
    myfunction('add1',10)
    print '\n END add'        


def subtract1():

  print '\n START Sub'  
  myfunction('sub1',100)   
  print '\n END Sub'


def main():

    t1 = threading.Thread(target=add1, args=('arg1','arg2'))
    t2 = threading.Thread(target=subtract1)
    t1.start()
    t2.start()


if __name__ == "__main__":
  main()

The output is as follows:

START add
START Sub
1. addition finish with Total:10
locked by add1
locked by add1
2. addition finish with Total:20
locked by add1
locked by add1
3. addition finish with Total:30 
locked by add1
 For add1() Total: 30
 END add
Subtraction finish with Total:-100
 For sub1() Total: -100
 END Sub

2. is it ok it we do not use locks?

Even if I do not use the lock mechanism defined in the above program the result is same from both threads t1 and t2. Does this mean that python implements locks automatically when multiple threads access the same function.

The output of the program without using the lock, functionLock , in the above program

START add
START Sub
1. addition finish with Total:10
Subtraction finish with Total:-100
For sub1() Total: -100
END Sub
2. addition finish with Total:20
3. addition finish with Total:30
For add1() Total: 30
END add

Thanks!

Answer 1

1. In your code you implement your own spin-lock. While this is possible, I don't think it's recommended in Python, since it might lead to a performance issue.

2. I used a well known searching engine (starts with G), querying about "python lock". On of the first results is this one: Thread Synchronization Mechanisms in Python. It looks like a good article to start with.

3. For the code itself: You should lock whenever the operation(s) executed on a shared resource are not atomic. It currently looks like there's no such resource in your code.

Answer 2

In addition to the other comments on this thread about busy waiting on a variable, I would like to point out that the fact that you are not using any kind of atomic swap may cause concurrency bugs. Even though your test execution does not cause them come up, if executed enough repetitions with different timings, the following sequence of events may come up:

Thread #1 executes while functionLock!='' and gets False. Then, Thread#1 is interrupted (preempted for something else to be executed), and Thread #2 executes the same line, while functionLock!='' also getting False. In this example, both threads have entered the critical section, which is clearly not what you wanted. In particular, in any line where threads modify total, the result may not be that which you expected, since both threads can be in that section at the same time. See the following example:

total is 10. For the sake of simplicity, assume num is always 1. Thread#1 executes total+=num, which is composed of three operations: (i) loading the value of total, (ii) adding it num and (iii) storing the result in total. If after (i), Thread#1 gets preempted and Thread#2 then executes total-=num, total is set to 9. Then, Thread#1 resumes. However, it had already loaded total = 10, so it adds 1 and stores 11 into the total variable. This effectively transformed the decrement operation by Thread#2 in a no-op.

Notice that in the wikipedia article linked by @ron-klein, the code uses an xchg operation, which atomically swaps a register with a variable. This is vital for the correction of the lock. In conclusion, if you want to steer clear of incredibly hard to debug concurrency bugs, never implement your own locks as alternative to atomic operations.

[edit] I just noticed that in fact total is a local variable in your code, so this could never happen. However, I believe that you are not aware that this is the cause of the code you have working perfectly, due to you affirming "Does this mean that python implements locks automatically when multiple threads access the same function.", which is not true. Please try adding global total to the beginning of myfunction, and executing the threads several times, and you should see errors in the output. [/edit]