I am curious to know what makes Robocopy (Robust File Copy) so fast and robust. Any body knows what is the API/Algo used for Robocopy? Anybody studied Robocopy?
I am asking since I have to write a method (in .NET/C#) which will copy directories/files fast and without errors... The amount of data can go up to 15Gb and I cannot simply call Robocopy for various reasons.
Thanks!
You can get very close to Robocopy's speed with a simple C# program that does asynchronous reads and writes using a standard FileStream with a 64K buffer. Larger buffer sizes up to 256K will give a slight performance increase. Larger than 256K will slow things down to a surprising extent. In my tests, using a 512K buffer took almost twice as long as copying with a 256K buffer.
The idea is pretty simple:
Read the first buffer from the source file
do
{
start asynchronous write to destination file.
Read the next buffer from the source file
wait for asynchronous write to complete
} while not end of file
It's a pretty simple thing to write. My program that does this is almost as fast as Robocopy and doesn't cause the kinds of problems that Robocopy causes when you're copying a very large (hundred gigabyte) file from a server.
A bit more info on the large file copy problem.
Note that this asynchronous read/write thing doesn't do much for performance if you're reading from and writing to the same physical disk. It's most effective when the source and destination are on different drives.