Split large file without copy?

Question

Question: Are there Windows API calls (perhaps NTFS only) which allows one to split a very large file into many others without actually copying any data (in other words, specify the logical breakpoints between joined files, with file names and sizes)?

Examples: SetFileValidData, NtSetInformationFile

Scenario: I need to programatically distribute/copy 10GB of files from a non-local drive (including network, USB and DVD drives). This is made up of over 100,000 individual files with median size about 16kbytes, but joined into ~2GB chunks.

However, using simple FileStream api's (64kb buffer) to extract files from the chunks on non-local drives to individual files on a local hard drive seems to be limited on my machine to about 4MB/s, whereas copying the entire chunks using Explorer occurs at over 80MB/s!

It seems logical to copy entire chunks, but give Windows enough info to logically split the files (which theoretically should be able to happen very, very fast).

Doesn't the Vista install do something like this?

Answer 1

Although there Volume Shadow Copies, these are an all-or-nothing approach - you can't cut out just part of a file. They are also only temporary. Likewise, hard links share all content, without exceptions. Unfortunately, cutting out just parts of a file is not supported on Windows, although some experimental Linux filesystems such as btrfs support it.