Python portable interprocess Semaphore/Event

Question

I'm creating a website using Flask. My WSGI server, Gunicorn, spawns multiple processes.

I have some cross-process objects (notably files) that I want to constrain access to within these processes, and raise events when they are modified.

The choice is normally to use system-wide mutexes/semaphores and events.

However, I can't find a portable (Windows/Mac/Linux) solution for these on Python.

The multiprocessing module (see this question), as far as I can tell, only works for processes spawned by the multiprocessing module itself, which these are not. There are POSIX semaphores also, but these only work on Linux.

Does anyone know of a more general solution?

Answer 1

I have been researching this for a while, and the closest I could find is the python file-locking library fasteners:

It works quite well in all platforms. The problem it only implements system mutex, but not semaphore like counting. I have implementing my own counting in a locked file with an integer counter and active waiting, but this is still fragile and will leave the system in bad state if one of the process crashes and doesn't update the count properly.