Using `splice` for Linux... what else for other systems?

Question

On recent Linux kernels, afaict, the fastest way of copying a file or a subset of a file to another file is through the use of the very nice splice system call. This system gets the kernel to manage the transfer (almost) directly, without ever having to copy the data to userland memory.

Now, I would be interested in finding something similar for other systems, in particular Windows and BSD/MacOS X (I am willing to get some code written in Obj-C in the unlikely case there is an API that cannot be accessed from raw C).

Thanks

Note I assume that the fastest full-file copy under Windows is CopyFile. But what about copying only a range of a file?

Note I am working on code that is both performance-sensitive and should work on a variety of mass storage devices (i.e. from Android smartphones to NFS mounts). I am, of course, willing to write different implementations for different OSes.

Note At the moment, I use splice under Linux/Android, copyfile under BSD/Mac OS X, CopyFile under Windows, and fallback to read/write for ranges in files.

Answer 1

In the BSDs sendfile() fills this role, as long as the destination file descriptor is a socket.

In all POSIX OSes, memmap() can be used to achieve this, by mapping a whole file, then calling write() on the mapped data. Since you never access this memory in your process, it can still have zero-copy performance.

Almost everywhere, the read()/write() combination works well, and is very fast. Zero-copy is cool, but with a page-sized buffer ~4096, you'll be rather close to the same speed, but with code everyone understands.