Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Rsync cronjob that will only run if rsync isn't already running

Tags:

I have checked for a solution here but cannot seem to find one. I am dealing with a very slow wan connection about 300kb/sec. For my downloads I am using a remote box, and then I am downloading them to my house. I am trying to run a cronjob that will rsync two directories on my remote and local server every hour. I got everything working but if there is a lot of data to transfer the rsyncs overlap and end up creating two instances of the same file thus duplicate data sent.

I want to instead call a script that would run my rsync command but only if rsync isn't running?

like image 381
mfpockets Avatar asked Feb 22 '12 06:02

mfpockets


People also ask

Will Cron start a new job if the current job is not complete?

They'll run at the same time. The standard practice around this is to create a file lock (commonly referred to as a flock), and if the lock isn't available, don't run.

Does rsync keep running?

The solution to keep rsync running in backgroundNohup allows to run a process/command or shell script to continue working in the background even if you close the terminal session. In our example, we also added '&' at the end, that helps to send the process to background.


1 Answers

The problem with creating a "lock" file as suggested in a previous solution, is that the lock file might already exist if the script responsible to removing it terminates abnormally. This could for example happen if the user terminates the rsync process, or due to a power outage. Instead one should use flock, which does not suffer from this problem.

As it happens flock is also easy to use, so the solution would simply look like this:

flock -n lock_file -c "rsync ..."

The command after the -c option is only executed if there is no other process locking on the lock_file. If the locking process for any reason terminates, the lock will be released on the lock_file. The -n options says that flock should be non-blocking, so if there is another processes locking the file nothing will happen.

like image 62
J. P. Petersen Avatar answered Sep 30 '22 00:09

J. P. Petersen