Running python script with cron only if not running

Question

I need to run a python script (job.py) every minute. This script must not be started if it is already running. Its execution time can be between 10 seconds and several hours.

So I put into my crontab:

* * * * * root cd /home/lorenzo/cron && python -u job.py 1>> /var/log/job/log 2>> /var/log/job/err

To avoid starting the script when it is already running, I use flock().

This is the script (job.py):

import fcntl
import time
import sys

def doIncrediblyImportantThings ():
    for i in range (100):
        sys.stdout.write ('[%s] %d.\n' % (time.strftime ('%c'), i) )
        time.sleep (1)

if __name__ == '__main__':
    f = open ('lock', 'w')
    try: fcntl.lockf (f, fcntl.LOCK_EX | fcntl.LOCK_NB)
    except:
        sys.stderr.write ('[%s] Script already running.\n' % time.strftime ('%c') )
        sys.exit (-1)
    doIncrediblyImportantThings ()

This approach seems to work.

Is there anything I am missing? Are there any troubles I can run into using this approach?

Are there more advised or "proper" ways of achieving this behaviour?

I thank you for any suggestion.

Answer 1

The only suggestion I would make is to make your exception handling a little more specific. You don't want to accidentally delete the fcntl import one day and hide the NameError that results. Always try to catch the most specific exception you want to handle. In this case, I suggest something like:

import errno

try:
    fcntl.lock(...)
except IOError, e:
    if e.errno == errno.EAGAIN:
        sys.stderr.write(...)
        sys.exit(-1)
    raise

This way, any other cause of the lock being unobtainable shows up (probably in your email since you're using cron) and you can decide if it's something for an administrator to look at, another case for the program to handle, or something else.

Answer 2

You're in trouble when the machine reboots or freezes with the script running (and thus an active lock). Simple way to counter this is to use the @reboot cron timestamp to run rm /path/to/lock.