Using python3 in shell script in crontab

Question

I try to run a shell script with crontab which runs python3 scripts. The crontab is for a user group. Now it runs the script but not the python3 scripts inside it. I try to debug it but I can't figure out what happens. It might be a permission issue or a path problem but I can't figure out. This is the line crontab

*/5 * * * * /home/group_name/path/to/script/run.sh

As I said the cron job is executed or at least thats what I think since when I run sudo grep CRON /var/log/syslog I get lines like

 Feb 16 20:35:01 ip-**-**-*-*** CRON[4947]: (group_name) CMD (/home/group_name/path/to/script/run.sh)

right below I also get a line which might have something to do with the problem

Feb 16 20:35:01 ip-**-**-*-*** CRON[4946]: (CRON) info (No MTA installed, discarding output)

and finally run.sh looks like this

#!/bin/bash

# get path to script and path to script directory
SCRIPT=$(readlink -f "$0")
SCRIPTPATH=$(dirname "$SCRIPT")

echo "set directory"
cd "$SCRIPTPATH"

echo "run first script"
/usr/bin/python3 ./first_script.py > ./log1.txt

However when the cron job executes it nothing happens, when I run it manually the cahnges to the database happen as expected. The group has the same rights as I have. The shell file can be executed by me and the group and the python files can't be executed by me so I don't know why the group would need this.

PS: I want to execute the python script in a shell since we have a lot of scripts with some times a lot of arguments and hence the crontab would become overpopulated and some scripts have to be executed in a certain order.

EDIT: Adding exec >> /tmp/output 2>&1 right after #! /bin.bash writes the echoes to /tmp/output whenever I run it manually but not when I run it in cron, not even the one before running any python script.

Running one of the python scripts directly from cron works, however even if I copy paste the exact same line as the one that works in cron, into the shell file, nothing happens.

Answer 1

There are a lot of components to this problem. I'm ignoring the MTA error because that's just about an email notification when your cron job finishes. I'll also assume you have permissions set properly and that your script runs fine when run manually in the shell.

The biggest issue is that the CRON command isn't the same as running the command from the terminal "shell". You have to specify that your script get run using bash. Change your cron job from:

*/5 * * * * /home/group_name/path/to/script/run.sh 

to:

*/5 * * * * bash /home/group_name/path/to/script/run.sh

This question has more information and additional options for solving the problem.

Answer 2

Change this line:

*/5 * * * * /home/group_name/path/to/script/run.sh

to:

*/5 * * * * cd /home/group_name/path/to/script && /home/group_name/path/to/script/run.sh

Regarding /var/log/syslog, when you look in /var/log/syslog, look at the timestamps to figure out if the cron job is being run.

Regarding the cron job not being able to write into log.txt, it could have to do with permissions. Try changing this line:

/usr/bin/python3 ./first_script.py > ./log1.txt

to:

/usr/bin/python3 /full/path/to/first_script.py > /tmp/log1.txt

See if there is any difference. cron should be able to write into /tmp.

Answer 3

1) The message "No MTA installed" does not mean any error, it only indicates that without mail server cron cannot report any details.

Modify cron job to log its output into syslog:

*/5 * * * * /home/group_name/path/to/script/run.sh 2>&1 | /usr/bin/logger -t run.sh

Then check results via sudo tail -f /var/log/syslog (or sudo tail -f /var/log/messages on RedHat and SuSE)

Alternatively, install Postfix and configure it for "local only" delivery:

sudo apt-get install postfix

Then check mail as the group_name user.

2) The redirection > ./log1.txt in run.sh should overwrite log file upon each execution. If python script failed with exception then log1.txt would remain truncated to zero length. In that case modify the last line of run.sh:

/usr/bin/python3 ./first_script.py 2>&1 > ./log1.txt

and check results.

If log1.txt is neither truncated nor contains fresh output then the python script is not being launched at all. Refer to step 1) to debug the run.sh.

Answer 4

The last line in your bash script contains relative paths (./) I believe this is the problem