Cronjob – how to do it the right way?

Question

A very common need for an application is to run a script every X minutes/hours. Basically its nothing complicated, just some PHP code and a crontab entry.

Although I've written quite a few of those cronjobs in the past years I still haven't seen any best practices, at least not that much. As with every "background processing" so many things can go wrong especially in a production settings.

Among them:

• an error occured during execution of the cron and the script died processing half of the data
• the cronjob was accidently started twice by another process/by user error/whatever
• the cronjob took way longer then expected and the script is called again although its not done processing data
• etc.

What are some best pratices for writing rock-solid, robust cronjob scripts? Writing a lock file asserting that only one instance runs, extensive logging and monitoring in oder to prevent sending ten thousands of duplicate emails? What are your ideas?

Answer 1

Personally, the way I handle errors is to simply send STDERR to a log file, and then periodically check that file. An easy way to do that, is to append 2>/pathtolog to the crontab entry.

As far as having duplicates of the same program running, I prefer to have the script attempt to lock something (a file or a local network port). If it fails to obtain that lock, the script does not run. This way, if an existing script is currently running, a new one cannot obtain the identical lock.