How can I prevent overlapping cron jobs with Rails?

Question

I have a cron job setup to run a task every 5 minutes. But sometimes the task takes > 5 minutes to run, so another copy of that task is run concurrently by cron. Is there a way in whenever or cron to we can make it to wait for the other job to finish before running another copy?

Answer 1

AFAIK You can't do this using whenever itself but you can handle this in your script. This can be done by one of the following solutions:

1. Handling this in the database using a flag(or some information like start time, end time, success status) that is set when the job starts and cleared when the job ends, and check this flag every time the job starts to see if the previous job is done or not. But make sure to handle exceptions as if the process dies before clearing the flag, no other process will be able to run.

2. You can cause the OS to work as a flag for you, by creating a temp file and having an exclusive lock on it for the current process so no other process can have an exclusive lock on this file until the current process finishes, then when the process finishes it'll release the lock and let other processes work. To do so include this in the top of your cron job:

   file = File.new("cron.lock", "a")
   can_lock = file.flock(File::LOCK_EX | File::LOCK_NB)
   
   if can_lock == false
     exit 1
   else
     #do whatever you want
   end
   

The advantage of the second method is that even if the process is terminated unexpectedly the lock will be released automatically by the OS.

For me, I chose the first method as I needed to start another process if the previous process finished or took longer than a specific time limit.