Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Where does the linux kernel keep the data written to a pipe

Tags:

linux

pipe

Suppose the following simple code:

#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>

main()
{
    int fd[2];                
    pipe(fd);
 // ...               
    write(fd, buf, VERY_BIG_NUMBER);
 // ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
}

Now my questions:

  1. Could somebody explain where the write will be putting the data?
  2. Is it written to some buffer located in user space or kernel space?
  3. What it's the maximum length that one could write to the pipe?
  4. Since the pipe write is blocking operation, wouldn't the kernel for efficiency just do nothing and wait for the reader then pass the data directly to it?
like image 770
rkachach Avatar asked Oct 15 '15 14:10

rkachach


1 Answers

There is a pipe buffer, typically 64KB on Linux. Writes up the pipe buffer. Once it's full the write() syscall blocks until the other end of the pipe drains the buffer.

Excerpt from the pipe(7) man page:

A pipe has a limited capacity. If the pipe is full, then a write(2) will block or fail, depending on whether the O_NONBLOCK flag is set (see below). Different implementations have different limits for the pipe capacity. Applications should not rely on a particular capacity: an application should be designed so that a reading process consumes data as soon as it is available, so that a writing process does not remain blocked.

In Linux versions before 2.6.11, the capacity of a pipe was the same as the system page size (e.g., 4096 bytes on i386). Since Linux 2.6.11, the pipe capacity is 65536 bytes.

The buffer is in kernel space. Pipes are implemented via a virtual pipefs filesystem. The kernel code for this filesystem allocates 16 4KB pages for each pipe2() system call, and that buffer space is associated with the inode created for the pipe. read() and write() syscalls copy data into and out of userspace. (Source)

Writes to a pipe are atomic if they're under 4KB. A write over 4KB could incur page faults and therefore is no longer atomic. That means that multiple writes from separate processes could be interleaved if the writes are larger than 4KB.

See also: How big is the pipe buffer?

like image 190
John Kugelman Avatar answered Oct 18 '22 15:10

John Kugelman