How to use a file as a mutex in Linux and C?

Question

I have different processes concurrently accessing a named pipe in Linux and I want to make this access mutually exclusive.

I know is possible to achieve that using a mutex placed in a shared memory area, but being this a sort of homework assignment I have some restrictions.

Thus, what I thought about is to use locking primitives on files to achieve mutual exclusion; I made some try but I can't make it work.

This is what i tried:

flock(lock_file, LOCK_EX)

// critic section

flock(lock_file, LOCK_UN)

Different projects will use different file descriptors but referring to the same file. Is it possible to achieve something like that? Can you provide some example.

Answer 1

The standard lock-file technique uses options such as O_EXCL on the open() call to try and create the file. You store the PID of the process using the lock, so you can determine whether the process still exists (using kill() to test). You have to worry about concurrency - a lot.

Steps:

• Determine name of lock file based on name of FIFO
• Open lock file if it exists
• Check whether process using it exists
  • If other process exists, it has control (exit with error, or wait for it to exit)
  • If other process is absent, remove lock file
• At this point, lock file did not exist when last checked.
• Try to create it with open() and O_EXCL amongst the other options.
• If that works, your process created the file - you have permission to go ahead.
• Write your PID to the file; close it.
• Open the FIFO - use it.
• When done (atexit()?) remove the lock file.

Worry about what happens if you open the lock file and read no PID...is it that another process just created it and hasn't yet written its PID into it, or did it die before doing so? Probably best to back off - close the file and try again (possibly after a randomized nanosleep()). If you get the empty file multiple times (say 3 in a row) assume that the process is dead and remove the lock file.

You could consider having the process that owns the file maintain an advisory lock on the file while it has the FIFO open. If the lock is absent, the process has died. There is still a TOCTOU (time of check, time of use) window of vulnerability between opening the file and applying the lock.

Take a good look at the open() man page on your system to see whether there are any other options to help you. Sometimes, processes use directories (mkdir()) instead of files because even root can't create a second instance of a given directory name, but then you have issues with how to know the PID of the process with the resource open, etc.

Answer 2

I'd definitely recommend using an actual mutex (as has been suggested in the comments); for example, the pthread library provides an implementation. But if you want to do it yourself using a file for educational purposes, I'd suggest taking a look at this answer I posted a while ago which describes a method for doing so in Python. Translated to C, it should look something like this (Warning: untested code, use at your own risk; also my C is rusty):

// each instance of the process should have a different filename here
char* process_lockfile = "/path/to/hostname.pid.lock";
// all processes should have the same filename here
char* global_lockfile = "/path/to/lockfile";
// create the file if necessary (only once, at the beginning of each process)
FILE* f = fopen(process_lockfile, "w");
fprintf(f, "\n"); // or maybe write the hostname and pid
fclose(f);

// now, each time you have to lock the file:
int lock_acquired = 0;
while (!lock_acquired) {
    int r = link(process_lockfile, global_lockfile);
    if (r == 0) {
        lock_acquired = 1;
    }
    else {
        struct stat buf;
        stat(process_lockfile, &buf);
        lock_acquired = (buf.st_nlink == 2);
    }
}
// do your writing
unlink(global_lockfile);
lock_acquired = 0;

Answer 3

Your example is as good as you're going to get using flock (2) (which is after all, merely an "advisory" lock (which is to say not a lock at all, really)). The man page for it on my Mac OS X system has a couple of possibly important provisos:

Locks are on files, not file descriptors. That is, file descriptors duplicated through dup(2) or fork(2) do not result in multiple instances of a lock, but rather multiple references to a single lock. If a process holding a lock on a file forks and the child explicitly unlocks the file, the parent will lose its lock

and

Processes blocked awaiting a lock may be awakened by signals.

both of which suggest ways it could fail.

---

// would have been a comment, but I wanted to quote the man page at some length