Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Consume stdin multiple times in bash script

Tags:

bash

tee

My script accepts a stream on stdin. I want to pass the first line through to stdout no matter what, and grep the remaining lines with -v and also pass those out to stdout.

I worked out a solution using tee, but I'm wondering if this is guaranteed to always print the output of head before the output of grep? What if head was replaced with something that blocked for 20 minutes before printing anything, would that output appear at the end of stdout after the output of grep?

tee >(head -n 1) >(tail -n +2 | grep -v -E "$PATTERN")

If the order is not guaranteed, what is the right way of doing this?

like image 256
Rob Crowell Avatar asked Mar 25 '14 21:03

Rob Crowell


Video Answer


2 Answers

You are overthinking this and you don't need tee, head or tail.

You can consume the first line using read and just print it out, then use grep on the rest:

$ printf "foo\nbar\nquux\n" | { read v; echo "$v"; grep -v bar; }
foo
quux

Or, combining the logic into a single awk statement and avoiding the problem altogether:

$ printf "foo\nbar\nquux\n" | awk 'NR==1{print;next} !/bar/'
foo
quux
like image 57
Adrian Frühwirth Avatar answered Sep 30 '22 09:09

Adrian Frühwirth


You're right to be paranoid. The two sub-shells will run in parallel, so there's no guarantee that one will run before the other. To force an order of operations, read and print the first line before you grep the rest of the input.

read line && printf '%s\n' "$line"
tee >(grep -v -E "$PATTERN")
like image 35
John Kugelman Avatar answered Sep 30 '22 08:09

John Kugelman