Why does the Python filelock library delete lockfiles on Windows but not UNIX?

Question

I'm using the filelock module for Python.

• On Windows, when a lock is released, the file that's backing it is deleted.

• On UNIX, lock files still exist on the filesystem even after the locks are released.

Is there a reason this is different between operating systems? If there isn't a reason for it to differ, which of these behaviors is more correct?

Answer 1

Speaking Specifically To py-filelock

The filelock library used to delete lockfiles on UNIX; this behavior was removed as of benediktschmitt/py-filelock#31, which refers to flock(): removing locked file without race condition? -- which discusses the same race condition described in a later section of this answer.

---

Why General Practices Differ

The operating-system semantics are different, so different approaches are appropriate in each case. In UNIX, you can delete a file even while there's an open handle, so lockfiles must not be deleted or else two programs can both think they hold the same lock, when in fact they hold locks on completely different inodes which were, at different points in time, referenced under the same filename.

By contrast, default filesystem semantics on Windows make it impossible to delete a file while any program has it open (even though NTFS is powerful enough to support it, it's artificially prevented for backwards compatibility with programs designed around FAT limitations), so on Windows, it's safe to delete a lockfile: If the deletion goes through, that proves that nobody held the lock (or was even in the process of opening the file to later grab a lock on it).

---

A Specific Race Example

To provide an example of how allowing open files to be unlinked on UNIX makes deleting lockfiles dangerous, consider the following illustration of a common race condition:

• Program 1 creates and opens file A1 (under the name "A"), receiving a file handle attached to the inode (the object reflecting the actual file itself, not the directory entry it's attached to) for that newly-created file.
• Program 1 requests an exclusive advisory lock on that handle. No other processes have a handle on that same file, so its request for a lock is granted.
• Program 2 opens file A1, receiving a second file handle on same.
• Program 2 requests an exclusive advisory lock on that handle. However, because Program A already holds a lock, the request blocks -- which is to say, the program waits for the operating system to pass control back to it later, when the lock can be granted.
• Program 1 finishes the processes it needed the lock for.
• Program 1 uses the unlink() syscall to delete the lockfile. (To be safe on UNIX, just leave this step out). This doesn't delete the file itself (the "inode") until no programs have it open, but it does immediately delete the link to that file from the directory that previously contained it.
• Program 1 closes its handle on the file, thus releasing its lock. The inode is not deleted, because Program 2 still holds a handle.
• Program 2 is granted the lock it's been waiting for on the file handle (on the now-unlinked file A1) it opened back before the deletion happened, and is able to resume execution.
• Program 3 creates and opens a new file A2 (with a new and distinct inode) under the same name "A" (which is available because A1's inode is no longer linked to that name), getting a file handle on same.
• Program 3 requests a lock on the file handle it owns. This is immediately granted, because A2 is a different file from A1, and the still-running Program 2 holds a lock on A1, not A2. Thus, we end up with two programs -- Program 2 and Program 3 -- thinking they hold the same lock.

Consequently, the above illustrates how on UNIX, deleting lockfiles allows race conditions wherein a lock can appear to be held by two programs at once.