crontab to run python file if not running already

Question

I want to execute my python file via crontab only if its down or not running already. I tried adding below entry in cron tab but it does not work

24 07 * * * pgrep -f test.py || nohup python /home/dp/script/test.py & > /var/tmp/test.out

test.py works fine if i run pgrep -f test.py || nohup python /home/dp/script/test.py & > /var/tmp/test.out manually and it also works in crontab if i remove pgrep -f test.py || from my crontab and just keep 24 07 * * * nohup python /home/dp/script/test.py & > /var/tmp/test.out

Any idea why crontab does not work if i add pgrep -f? is there any other way i can run test.py just one time to avoid multiple running processes of test.py? Thanks, Deepak

Answer 1

pgrep -f lists itself as a false match when run from cron

I did the test with a script.py running an infinite loop. Then

pgrep -f script.py

...from the terminal, gave one pid, 13132 , while running from cron:

pgrep -f script.py > /path/to/out.txt

outputs two pids, 13132 and 13635.

We can therefore conclude that the command pgrep -f script.py lists itself as a match, when run from cron. Not sure how and why, but most likely, this is indirectly caused by the fact that cron runs with a quite limited set of environment variables (HOME, LOGNAME, and SHELL).

The solution

Running pgrep -f from a (wrapper) script makes the command not list itself, even when run from cron. Subsequently, run the wrapper from cron:

#!/bin/bash

if ! pgrep -f 'test.py'
then
    nohup python /home/dp/script/test.py & > /var/tmp/test.out
# run the test, remove the two lines below afterwards
else
    echo "running" > ~/out_test.txt
fi

Answer 2

It is generally a good idea to check whether the application is running or not within the application rather than checking from outside and starting it. Managing the process within the process rather than expecting another process to do it.

1. Let the cron run the application always
2. At the start of execution of the application, have a lock file or similar mechanism that will tell whether the process is already running or not.
3. If the application is already running, update somewhere the last test time and break the new process.
4. If the application is not running, log somewhere and notify someone (if required) and trigger the process start.

This will ensure that you will have more control over the life time of a process AND let you decide what to do in case of failures.

Not to mention, if you want to make sure that the application up time is to be high, it is best to make use of monitoring services such as Monit. Again this will depend on your application to add a sanity layer that says whether it is OK or not.