Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Pipe buffer size is 4k or 64k?

I read in multiple places that the default buffer size for a pipe is 4kB (for instance, here), and my ulimit -a tends to confirm that statement:

$ ulimit -a core file size          (blocks, -c) 0 data seg size           (kbytes, -d) unlimited scheduling priority             (-e) 0 file size               (blocks, -f) unlimited pending signals                 (-i) 15923 max locked memory       (kbytes, -l) 64 max memory size         (kbytes, -m) unlimited open files                      (-n) 1024 pipe size            (512 bytes, -p) 8 // 8 * 512B = 4kB POSIX message queues     (bytes, -q) 819200 real-time priority              (-r) 0 stack size              (kbytes, -s) 10240 cpu time               (seconds, -t) unlimited max user processes              (-u) 1024 virtual memory          (kbytes, -v) unlimited file locks                      (-x) unlimited 

But when I use a little program to test the buffer size (by writing into the pipe until the write() blocks), I see a limit of 64kB!

See this program:

#include <stdio.h> #include <unistd.h> #include <limits.h>  int main(void) {     int tube[2];     char c = 'c';     int i;      fprintf(stdout, "Tube Creation\n");     fprintf(stdout, "Theoretical max size: %d\n", PIPE_BUF);     if( pipe(tube) != 0)     {         perror("pipe");         _exit(1);     }     fprintf(stdout, "Writing in pipe\n");     for(i=0;; i++)     {         fprintf(stdout, "%d bytes written\n", i+1);         if( write(tube[1], &c, 1) != 1)         {             perror("Write");             _exit(1);         }     }     return 0; } 

And its output:

$ ./test_buf_pipe  Tube Creation Theoretical max size: 4096 Writing in pipe 1 bytes written 2 bytes written 3 bytes written 4 bytes written [...] 65535 bytes written [blocks here] 

It strongly suggests that the pipe buffer size is actually 64k! What is happening here??

like image 913
Gui13 Avatar asked Jan 07 '11 09:01

Gui13


People also ask

How big is the pipe buffer?

Since Linux 2.6. 11, the pipe capacity is 16 pages (i.e., 65,536 bytes in a system with a page size of 4096 bytes).

What happens when a process executes a write () to a pipe with a full buffer?

When a process uses WriteFile to write to an anonymous pipe, the write operation is not completed until all bytes are written. If the pipe buffer is full before all bytes are written, WriteFile does not return until another process or thread uses ReadFile to make more buffer space available.


2 Answers

The other answers tell you that the pipe size is 64 KB. The reason why PIPE_BUF is 4KB is that PIPE_BUF is the largest size for which writes are guaranteed to be atomic. See http://pubs.opengroup.org/onlinepubs/9699919799/functions/write.html

like image 93
janneb Avatar answered Sep 22 '22 18:09

janneb


It's programmable now


As of Linux 2.6.35 you can use fcntl(2) with the F_SETPIPE_SZ operation to set the pipe buffer up to /proc/sys/fs/pipe-max-size. This is by default 1 MB; see proc(5).

like image 43
DigitalRoss Avatar answered Sep 23 '22 18:09

DigitalRoss