is there any sync algorithm/reference available for syncing a directory?

Question

I'm planning to write a program to sync a folder in real time across multiple computers over the internet. I'm wondering if there is any sync algorithm to handle file sync conflicts, ie, computer A tries to save a file, while computer B has removed the file.

Answer 1

The example you gave is exactly why synchronization is considered a hard problem.

Computer A has deleted a file which computer B still has. Now: how do you know if the file was added on B, and should be copied to A, or deleted on A, and should be deleted on B? You don't, really. Many synchronization systems have the possibility of conflicting changes which need to be resolved by a human.

Many tools have already been built to do synchronization, including:

• version control systems, like CVS, Subversion, Mercurial, git, Perforce, etc.
• standalone, one-directional synchronization programs. They cannot handle changes on either side, but they can make the destination directory look exactly like the source directory. This is better than a complete COPY because it's faster, but it's really basically the same thing. Examples include rsync, ROBOCOPY and XCOPY /MIR on Windows.
• easy to use internet synchronization tools that synchronize folders on multiple machines. Examples include Windows Live Folder and Dropbox. These apps often resolve conflicts by making extra copies of both versions in subdirectories so that you can sort it out later. They really assume that there will be very very few conflicts.
• built-in synchronization in sophisticated applications, for example, email/contact/calendar synchronization in Microsoft Exchange, Lotus Notes, etc.

Answer 2

You might also want to look into Unison. It is a multidirectional file synchronization tool that uses the rsync algorithm to make sure that only the changed parts of files are sent.