Process substitution into grep missing expected outputs

Question

Let’s say I have a program which outputs:

abcd
l33t
1234

which I will simulate with printf 'abcd\nl33t\n1234\n'. I would like to give this output to two programs at the same time. My idea would be to use process substitution with tee. Let’s say I want to give a copy of the output to grep:

printf 'abcd\nl33t\n1234\n' | tee >(grep '[a-z]' >&2) | grep '[0-9]'

I get the following with Bash 4.1.2 (Linux, CentOS 6.5), which is fine:

l33t
1234
abcd
l33t

But if the process substitution is not redirected to stderr (i.e. without >&2), like this:

printf 'abcd\nl33t\n1234\n' | tee >(grep '[a-z]') | grep '[0-9]'

Then I get:

l33t
1234
l33t

It’s like the stdout from process substitution (the first grep) is used by the process after the pipe (the second grep). Except the second grep is already reading things by itself, so I guess it's not supposed to take into account things from the first grep. Unless I’m mistaken (which I surely am).

What am I missing?

Answer 1

Explanation

As far as the command line is concerned, process substitution is just a way of making a special filename. (See also the docs.) So the second pipeline actually looks like:

printf 'abcd\nl33t\n1234\n' | tee /dev/fd/nn | grep '[0-9]'

where nn is some file-descriptor number. The full output of printf goes to /dev/fd/nn, and also goes to the grep '[0-9]'. Therefore, only the numerical values are printed.

As for the process inside the >(), it inherits the stdout of its parent. In this case, that stdout is inside the pipe. Therefore, the output of grep '[a-z]' goes through the pipeline just like the standard output of tee does. As a result, the pipeline as a whole only passes lines that include numbers.

When you write to stderr instead (>&2), you are bypassing the last pipeline stage. Therefore, the output of grep '[a-z]' on stderr goes to the terminal.

A fix

To fix this without using stderr, you can use another alias for your screen. E.g.:

printf 'abcd\nl33t\n1234\n' | tee >(grep '[a-z]' >/dev/tty ) | grep '[0-9]'
                                               # ^^^^^^^^^

which gives me the output

l33t
1234
abcd
l33t

Testing this

To sort this out, I ran echo >(ps). The ps process was a child of the bash process running the pipeline.

I also ran

printf 'abcd\nl33t\n1234\n' | tee >(grep '[a-z]')

without the | grep '[0-9]' at the end. On my system, I see

abcd    <--- the output of the tee
l33t         ditto
1234         ditto
abcd    <--  the output of the grep '[a-z]'
l33t         ditto

All five lines go into the grep '[0-9]'.