Are Unix/Linux pipes producer or consumer driven?

Question

Suppose I have this:

A | B | C

How does the pipeline work? Does A produce data only when B requests it? Does A continually produce data and then block if B can't currently accept it? What's C's role? I realized that a system I'm designing is conceptually very similar to these pipelines -- I'd like to draw upon the existing paradigm rather than inventing something novel that only works half as well.

Answer 1

Pipes in Unix have a buffer, so even if the right side process (RSP) does not consume any data, the left side process (LSP) is able to produce a few kilobytes before blocking.

Then, if the buffer gets full, the LSP is eventually blocked. When the RSP reads data it frees part or all of the buffer space and the LSP resumes the operation.

If instead of 2 processes you have 3, the situation is more or less the same: a faster producer is blocked by a slower consumer. And obviously, a faster consumer is blocked by a slower producer if the pipe gets empty: just think of an interactive shell, waiting of the slowest producer of all: the user.

For example the following command:

$ yes | cat | more

Since more blocks when the screen is full, until the user presses a key, the cat process will fill its output buffer and stall, then the yes process will fill its buffer and also stall. Everything waiting for the user to continue, as it should be.

PS: As an interesting fact is: what happens when the more process ends? well, the right side of that pipe is closed, so the cat process will get a SIGPIPE signal (if it ever writes again in the pipe, and it will) and will die. The same will happen to the yes process. All processes die, as it should be.