Make this directory sync script detect change and run in the background

Question

I have this simple python script which will synchronize the contents of sourcedir folder to targetdir folder.

Here is the code;

from dirsync import sync

sourcedir = "C:/sourcedir"
targetdir ="C:/targetdir"
sync(sourcedir, targetdir, "sync")

It is cumbersome to manually run this script whenever changes are made. I would like to have this script running in the background so that whenever there is any change in sourcedir folder, targetdir folder will be synchronized automatically.

I am using python v3.5

Answer 1

There's an app a library for that:

import sys
import time
import logging
from watchdog.observers import Observer


def event_handler(*args, **kwargs):
    print(args, kwargs)


if __name__ == "__main__":
    path = '/tmp/fun'
    observer = Observer()
    observer.schedule(event_handler, path, recursive=True)
    observer.start()
    try:
        while True:
            time.sleep(1)
    except KeyboardInterrupt:
        observer.stop()
    observer.join()

Answer 2

If you're running your script on linux, you can use inotify. (GitHub).

It uses a kernel feature that notifies an event when something happens in a watched directory, as file modification, access, creation, etc. This has very little overhead, as it's using epoll system call to watch for changes.

import inotify.adapters

i = inotify.adapters.Inotify()
i.add_watch(b'/tmp')
try:
    for event in i.event_gen():
        if event is not None:
            (header, type_names, watch_path, filename) = event
            if 'IN_MODIFY' in type_names:
                # Do something
                sync(sourcedir, targetdir, "sync")
finally:
    i.remove_watch(b'/tmp')

Also, it's recommended to use multiprocessing to execute the sync part, unless the script will not watch for changes during the sync process. Depending on your sync implementation, this could lead to process synchronization problems, a huge topic to discuss here.

My advice, try the easy approach, running everything on the same process and test if it suits your needs.