Solution for running scheduled cron job within a Docker container?

Question

I have a Docker container in which I have my Python tools installed, including my Luigi pipeline interface. I would like to run a shell script which kicks off my Luigi pipeline on a weekly basis using cron.

I have tried high and low to get cron to work within a Docker container. I cannot, for the life of me, get my crontab -e file to run.

In my file I have:

0 0 * * Sun /data/myscript.sh

followed by a new line. Cron is running in the background - ps aux | grep cron shows /usr/sbin/cron is running. Furthermore, in my /var/log/syslog file, I have:

/USR/SBIN/CRON[2037]: (root) CMD (/data/myscript.sh)

I've also tried using 0 0 * * Sun . /root/.bashrc ; sh /data/myscript.sh

However, my script does not run (when I run my script manually using bash myscript.sh, I get the expected results).

Suggestions?

Answer 1

Scheduled tasks won't run inside of a normal container since there is no scheduler running. The only active task will be that which you have elected to run via the CMD keyword or the Entrypoint.

In order to execute schedule tasks, it's more prudent to utilize the host scheduler and docker exec commands:

docker exec <container> <command>
docker exec <container> /data/myscript.sh

So you would end up with a cron on your host something like :

(Crontab Style) 0 * * * * root docker exec mycontainer /data/myscript.sh

If you have a cluster, you would have to query the cluster first to locate the container, or even have a script do it for you.