Is sed blocking?

Question

I had the impression sed wasn't blocking, because when I do say:

iostat | sed

sed processes the data as it arrives, but when I do

iostat | sed | netcat

Then sed blocks netcat.

Am I right?

Answer 1

sed will work in buffered mode when it doesn't print to a terminal. This means that it will try to fill its internal buffer before doing any processing and output by default.

This is done to increase throughput, because normally in a pipe you don't care about the timing, but want as much data processed in a given time as possible.

Passing -u to sed will tell it to work unbuffered, therefore working the same way it works when output goes to a terminal.

Answer 2

In addition to what @saua says, sed is at least line oriented, that it, it read a line, then operates on it so it will always be buffering at least one line. In addition, sed can work in multiline mode. If you are using a multiline pattern, then sed can't output it's current buffer until it knows that the pattern either doesn't apply or the pattern has been processed.

Answer 3

I don't know if I understand the question right, but in your example, it should be like this:

• sed waits for iostat to send data
• iostat will wait for sed to process the data if sed is slow
• netcat waits for sed to send data
• sed will wait for netcat again, if it is slow

Other than that, sed shouldn't need to read all its input to produce output.

Do you observe any delays that cannot be explained by this and some small buffering?