I want to run 2 commands on a piped-in input and want to print (to stdout) the output of both.
Each command is a combination of grep, sed and awk.
Both these commands must reside in a single .sh file.
Sample commands:
cat mult_comm.sh
sed 's/World/Boy/g'|grep Boy ; grep World
# Input
cat input.log
Hello World
# This command HAS to work EXACTLY like this
cat input.log | bash mult_comm.sh
Expected output
Hello Boy
Hello World
Actual output
Hello Boy
I tried using tee
cat mult_comm.sh
tee >(sed 's/World/Boy/g'|grep Boy) | grep World
But this gives only
Hello World
I can modify the .sh file as I want but the piped command can't be changed. Any ideas?
This is similar to OS X / Linux: pipe into two processes? and Pipe output to two different commands, but I can't figure out how to use named pipes inside the script.
When you execute
tee >(some_command)
bash creates a subshell to run some_command
. The subshell's stdin
is assigned to the reading half of a pipe. bash leaves the name of this pipe on the command line, so that tee
will pump its input into the pipe. The subshell's stdout
and stderr
are left unchanged, so they are still the same as tee
's.
So, when you execute
tee >(some_command) | some_other_command
Now, bash first creates a process to run tee
, and assigns its stdout
to the writing half of a pipe, and another process to run some_other_command
, with its stdin
assigned to the reading half of the same pipe. Then it creates another process to run some_command
, as above, assigning its stdin
to the reading half of another pipe, and leaving its stdout
and stderr
unchanged. However, stdout
has already been redirected to some_other_command
, and that's what some_command
inherits.
In your actual example,
tee >(sed 's/World/Boy/g'|grep Boy) | grep World
we end up with:
--> sed 's/World/Boy/g' --> grep Boy --
/ \
input --> tee --< \
\ \
----------------------------------------------> grep World
In one of the questions linked in the OP, there is a (non-accepted but correct) answer by F. Hauri, which I've adapted here:
echo Hello World |
((tee /dev/fd/5 | grep World >/dev/fd/4) \
5>&1 | sed 's/World/Boy/' | grep Boy) 4>&1
It takes a little practice to read bashisms like the above. The important part is that
( commands ) 5>&1
Creates a subshell (( )
) and gives that subshell an fd numbered 5, initially copied from stdout
(5>&1
). Inside the subshell, /dev/fd/5
refers to that fd. Within the subshell, it is possible to redirect stdout
, but that will happen after stdout is copied to fd5.
You can use pee(1)
– in Debian/Ubuntu it's available in the package moreutils
.
Usage for your example, somewhat more readable than the magic redirection
echo Hello World | pee 'grep World' 'sed "s/World/Boy/" | grep Boy'
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With