Implementing distributed lock using files

Question

I have a network drive (Z:\) which is shared by multiple Windows computers. Is it possible to implement a cross-machine lock by simply creating/deleting files on this network drive?

For example, two computers, A and B, want to write to a shared resource with an ID of 123 at the same time.

One of the computers, say A, locks the resource first by creating an empty file Z:\locks\123. When B sees there is the lock file with the name of "123", B knows the resource 123 is being used by someone else, so it has to wait for Z:\locks\123 to be deleted by A before it can access the resource.

It's like critical section in multithreading, but I want to do it on multiple machines.

I'm trying to implement in Python. Here's what I came up with:

import os
import time


def lock_it(lock_id):

    lock_path = "Z:\\locks\\" + lock_id
    while os.path.exists(lock_path):
        time.sleep(5)  # wait for 5 seconds

    # create the lock file
    lock_file = open(lock_path, "w")
    lock_file.close()


 def unlock_it(lock_id):

     # delete the lock file
     lock_path = "Z:\\locks\\" + lock_id
     if os.path.exists(lock_path):
         os.remove(lock_path)

This won't work because there could be more than one processes exit the waiting status and create the lock file at the same time.

So again, the question is: Is it possible to implement a cross-machine locking mechanism on a shared storage?

Answer 1

... sort of.

First, you should create a lock directory instead of a lock file. Creating a directory (see os.mkdir) will fail if the directory already exists, so you can acquire the lock like this:

while True:
    try:
        os.mkdir(r"z:\my_lock")
        return
    except OSError as e:
        if e.errno != 21: # Double check that errno will be the same on Windows
            raise
        time.sleep(5)

Second (and this is where the "sort of" comes in) you'll want some way to notice when the person holding the lock has died. One simple way to do this might be to have them occasionally update a file inside the lock directory. Then if clients notice that file hasn't been updated in a while, they can remove the directory and try to acquire the lock themselves.

Answer 2

This will not work nearly as well you as might hope. You'll have other issues such as the network drive going away, in which case all your processes will either be stuck or think that no one is holding a lock.

I suggest you look into something like ZooKeeper. You will be able to create synchronous locks and recover in the event of network failures. The framework behind distributed locks is much more complex then creating a file on a network drive.