Python: Lock directory

Question

AFAIK this code can be used to lock a directory:

class LockDirectory(object):
    def __init__(self, directory):
        assert os.path.exists(directory)
        self.directory = directory

    def __enter__(self):
        self.dir_fd = os.open(self.directory, os.O_RDONLY)
        try:
            fcntl.flock(self.dir_fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
        except IOError as ex:
            if ex.errno != errno.EAGAIN:
                raise
            raise Exception('Somebody else is locking %r - quitting.' % self.directory)

    def __exit__(self, exc_type, exc_val, exc_tb):
        self.dir_fd.close()

But according to the answers of this question locking a directoy is not possible: Python: Lock a directory

What is wrong with above code?

I only need to support current linux version. No Windows, Mac or other unix.

Answer 1

I change your code a bit,add return self like most context manage do,then with dup(),the second context manage will fail.and the solution is simple,uncommentfcntl.flock(self.dir_fd,fcntl.LOCK_UN)

The mode used to open the file doesn't matter to flock.

and you cannot flock on NFS.

import os
import fcntl
import time
class LockDirectory(object):
    def __init__(self, directory):
        assert os.path.exists(directory)
        self.directory = directory

    def __enter__(self):
        self.dir_fd = os.open(self.directory, os.O_RDONLY)
        try:
            fcntl.flock(self.dir_fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
        except IOError as ex:             
            raise Exception('Somebody else is locking %r - quitting.' % self.directory)
        return self

    def __exit__(self, exc_type, exc_val, exc_tb):
        # fcntl.flock(self.dir_fd,fcntl.LOCK_UN)
        os.close(self.dir_fd)

def main():
    with LockDirectory("test") as lock:
        newfd = os.dup(lock.dir_fd)
    with LockDirectory("test") as lock2:
        pass

if __name__ == '__main__':
    main()

Answer 2

If all you need is a read lock, then there is only a minor error in the code you have. It is perfectly feasible to get a read lock on a directory.

You'll need to alter your __exit__ function to use os.close() to close the file descriptor; a file descriptor is just an integer, and integers have no .close() method:

def __exit__(self, exc_type, exc_val, exc_tb):
    os.close(self.dir_fd)

The usual confusion for people that think you can't, are those that have tried with the open() function. Python won't let you open a directory node with that function because there is no point in creating a Python file object for a directory. Or perhaps there is an assumption that you wanted the OS to enforce access to the directory via the lock (as opposed to an advisory lock that a cooperative set of processes agree to obtain first before attempting access).

So no, there is nothing wrong with the code if all you want is an advisory lock, and are fine with this only working on Linux.

I'd drop the directory distinction from the code. The lock will work on any path that you have read access to. It is not exclusive to directories.

The downside of locking the directory is that this doesn't give you a place to store lock metadata. While lsof can give you the PID of the current owner of the lock, you may want to communicate some other information with the lock to help troubleshoot or automate lock breaking. A .lock file or symlink would let you record additional information. For example, Mercurial will create a symlink with the hostname, the PID namespace identifier (Linux only) and the PID in the target name; you can create such a symlink atomically, while writing that data to a file would require creating a file under a temp name followed by a rename.