Duplicate stdout, pipe it to two different commands, collect results from both to stdin of final program

Question

Say I have three programs: generator, that produces input data fed to processor and verifier that can check if processor output is correct for given input (so it needs both files).

What I currently do is:

generator > in.txt && processor < in.txt > out.txt && cat in.txt out.txt | verifier

Is it possible to achieve the same result without using explicit files? I've read about duplicating input using tee and and process substitution, but I didn't find a way to collect both streams into single one for final step.

Answer 1

I have not tested this, but try:

{ generator | tee /dev/stderr | processor ; } 2>&1  | verifier

This will redirect a copy of generator output to stderr. Then run processor on stdout of generator. Then combine both & pipe to verifier.

However, this cannot guarantee the order, in which lines from generator & processor would reach verifier.

---

Alternately, you can try process substitution like below:

( generator | tee >(processor) ) | verifier

Answer 2

If you don’t want to create real files on your slow hard-disk, you can use FIFOs (First In First Out), which are also called named pipes, because of their behaviour.

mkfifo myfifo
generator | tee myfifo | processor | verifier myfifo

This streams the generated content to tee, which duplicates it to myfifo and to stdout, which is piped through the processor to the verifier. And the verifier also gets the stream from myfifo.