What's the algorithm behind Robocopy?

Question

I am curious to know what makes Robocopy (Robust File Copy) so fast and robust. Any body knows what is the API/Algo used for Robocopy? Anybody studied Robocopy?

I am asking since I have to write a method (in .NET/C#) which will copy directories/files fast and without errors... The amount of data can go up to 15Gb and I cannot simply call Robocopy for various reasons.

Thanks!

Answer 1

You can get very close to Robocopy's speed with a simple C# program that does asynchronous reads and writes using a standard FileStream with a 64K buffer. Larger buffer sizes up to 256K will give a slight performance increase. Larger than 256K will slow things down to a surprising extent. In my tests, using a 512K buffer took almost twice as long as copying with a 256K buffer.

The idea is pretty simple:

Read the first buffer from the source file
do
{
    start asynchronous write to destination file.
    Read the next buffer from the source file
    wait for asynchronous write to complete
} while not end of file

It's a pretty simple thing to write. My program that does this is almost as fast as Robocopy and doesn't cause the kinds of problems that Robocopy causes when you're copying a very large (hundred gigabyte) file from a server.

A bit more info on the large file copy problem.

Note that this asynchronous read/write thing doesn't do much for performance if you're reading from and writing to the same physical disk. It's most effective when the source and destination are on different drives.