Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Copy a file without using the windows file cache

Anybody know of a way to copy a file from path A to path B and suppressing the Windows file system cache?
Typical use is copying a large file from a USB drive, or server to your local machine. Windows seems to swap everything out if the file is really big, e.g. 2GiB. Prefer example in C#, but I'm guessing this would be a Win32 call of some sort if possible.

like image 522
Ruvan Fernando Avatar asked Sep 07 '08 18:09

Ruvan Fernando


People also ask

How do I copy files without interruption?

Go to Windows File Explorer, select the files from the source folder, right-click and choose Copywhiz–>Copy as shown below: Go to the destination folder, right-click and choose Copywhiz–> Paste Advanced. The advanced settings dialogue box will open. Select the “General” tab and enable the option “Run silently”.

Does Windows have a file cache?

By default, Windows caches file data that is read from disks and written to disks. This implies that read operations read file data from an area in system memory known as the system file cache, rather than from the physical disk.

How do I duplicate a file in Windows?

There are many ways to copy a file in Windows. The simplest way is by using the Copy command which can be accessed by right-clicking on a file and selecting the command from the context menu. You can then left click somewhere else in the window so that the file is no longer selected, right click, and select Paste.


3 Answers

In C# I have found something like this to work, this can be changed to copy directly to destination file:

    public static byte[] ReadAllBytesUnbuffered(string filePath)
    {
        const FileOptions FileFlagNoBuffering = (FileOptions)0x20000000;
        var fileInfo = new FileInfo(filePath);
        long fileLength = fileInfo.Length;
        int bufferSize = (int)Math.Min(fileLength, int.MaxValue / 2);
        bufferSize += ((bufferSize + 1023) & ~1023) - bufferSize;
        using (var stream = new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.None,
                                           bufferSize, FileFlagNoBuffering | FileOptions.SequentialScan))
        {
            long length = stream.Length;
            if (length > 0x7fffffffL)
            {
                throw new IOException("File too long over 2GB");
            }
            int offset = 0;
            int count = (int)length;
            var buffer = new byte[count];
            while (count > 0)
            {
                int bytesRead = stream.Read(buffer, offset, count);
                if (bytesRead == 0)
                {
                    throw new EndOfStreamException("Read beyond end of file EOF");
                }
                offset += bytesRead;
                count -= bytesRead;
            }
            return buffer;
        }
    }
like image 73
nietras Avatar answered Oct 22 '22 05:10

nietras


Even more important, there are FILE_FLAG_WRITE_THROUGH and FILE_FLAG_NO_BUFFERING.

MSDN has a nice article on them both: http://support.microsoft.com/kb/99794

like image 38
gabr Avatar answered Oct 22 '22 06:10

gabr


I am not sure if this helps, but take a look at Increased Performance Using FILE_FLAG_SEQUENTIAL_SCAN.

SUMMARY

There is a flag for CreateFile() called FILE_FLAG_SEQUENTIAL_SCAN which will direct the Cache Manager to access the file sequentially.

Anyone reading potentially large files with sequential access can specify this flag for increased performance. This flag is useful if you are reading files that are "mostly" sequential, but you occasionally skip over small ranges of bytes.

like image 36
Espo Avatar answered Oct 22 '22 05:10

Espo