How is splice() actually doing zero-copy in Linux?

Question

I'm new to the concept of zero-copy but from what I understand, it is a way not to copy anything from kernel buffers to user buffers and pass data directly between the 2 kernel buffers. In this way, the CPU does not have to do 2 copies of data from the kernel buffer to user buffer and back to kernel buffer. All the CPU does now is it copies the data between the 2 kernel buffers, thereby reducing the no. of copies done by the CPU to 1. In some cases with Linux 2.4 and above, even the data doesn't have to be duplicated in the kernel buffers, only the location and length of data to be transferred are passed to the socket buffer and DMA does the copying. Hence the name zero-copy.

Two ways to do zero-copy in Linux are via sendfile() or via splice() syscalls.

While sendfile() has the inherent limitation of copying data only from the page cache of the file to the socket buffer, splice() on the other hand has no such limitation. But the problem is that in splice() either of the file descriptors should be a pipe. So the kernel has to first copy the data from the source file descriptor to the pipe and then copy the data back from the pipe to the destination kernel buffer. The number of copies by the CPU involved here is 2.

So my questions are:

• How is then splice() solving our original problem of reducing the number of copies done by CPU?
• Is zero-copy only possible between socket and file and vice versa and not file to file or socket to socket?

Answer 1

If you look at the NOTES section of the manual of splice(), it is said that "actual copies are generally avoided":

Though we talk of copying, actual copies are generally avoided. The kernel does this by implementing a pipe buffer as a set of reference-counted pointers to pages of kernel memory. The kernel creates "copies" of pages in a buffer by creating new pointers (for the output buffer) referring to the pages, and increasing the reference counts for the pages: only pointers are copied, not the pages of the buffer.

Concerning sendfile(), the manual specifies that "the out fd must be a socket" is no longer true since Linux 2.6.33:

In Linux kernels before 2.6.33, out_fd must refer to a socket. Since Linux 2.6.33 it can be any file. If it is a regular file, then sendfile() changes the file offset appropriately.