How to enforce only one running instance of a process in python Django framework?

Question

I have a python Django manage command that should be called upon receiving an input file but this command is not safe for parallel calls. So an input file should be processed only and only when there is no other file being processed.

One solution that I have is to use a lock file. Basically, create a lock file at the start of the process and delete it at the end.

I'm worried that if the process crashes the lock file won't be deleted and consequently none of the other files would be processed until we manually remove that lock file.

The solution doesn't need to be specific for Django or even python, but what is the best practice to enforce that only one instance of this process is being run?

Answer 1

As KlausD mentions in his comment, the canonical (and language-agnostic) solution is to use a lock file containing the pid of the running process, so the code responsible for the lock acquisition can check if the process is still running.

An alternative solution if you use redis in your project is to store the lock in redis with a TTL that's a bit longer than the worst case runtime of the task. This makes sure the lock will be freed whatever, and also allow to easily share the lock between multiple servers if needed.

EDIT:

is it possible that the process crashes and another process pick up the same pid?

Yes, of course, and that's even rather likely (and this is an understatement) on a server running for month or more without reboot, and even more so if the server runs a lot of short-lived processes. You will not only have to check if there's a running process matching this pid but also get the process stats to inspect the process start time, the command line, the parent etc and decides the likelyhood it's the same process or a new one.

Note that this is nothing new - most process monitoring tools face the same problem, so you may want to check how they solved it (gunicorn might be a good starting point here).