Don't think this should make a difference but I'll include it anyway
GNU bash, version 3.2.51
So say i have a pipe with multiple parts, how can I prevent part of the pipe running before the previous part is finished.
In the below examples I will try to display the issue
$ echo hello | tee /dev/tty | (echo "Next";sed 's/h/m/' )
Output
Next
hello
mello
With sleep to show that it is the timing that is off
$ echo hello | tee /dev/tty | (sleep 2;echo "Next";sed 's/h/m/' )
Output
hello
Next
mello
Is as above
hello
Next
mello
but obviously this depends on the sleep being longer than it takes for the previous command to finish which is not what i would want.
I know there are better ways to do this but I just think it would be educational for me to understand exactly how the pipe is working.
Tried variations of wait and sleep and things but nothing works consistently.
l0b0's suggestion
This still prints Next first though
$ echo hello | tee /dev/tty | sort |(echo "Next";sed 's/h/m/' )
Next
hello
mello
$ echo hello | tee /dev/tty | tac | tac |(echo "Next";sed 's/h/m/' )
Next
hello
mello
If any more information is needed then please let me know.
The point of a pipe is to process data asynchronously, so as to save time and space overall. If you want to have a synchronous pipe you might as well write to file (on a RAM disk if you need the speed). But for tasks where the receiving commands are able to handle data in chunks the complete pipeline may be much slower:
a | b | c
can at best be as fast as the slowest of the three commands.a > file; b < file > file2; c < file2
can at best be as fast as the sum of the runtimes of each command.So if the commands all run in about N seconds (when run separately), you're looking at a best case runtime of N for the first command and 3N for the second command.
There is no language construct in bash
to modify the behavior of a pipeline like you want. However, you can use a named pipe to act as a type of binary semaphore:
mkfifo block
echo hello |
{ tee /dev/tty; echo go > block; } |
(read < block; echo "Next"; sed 's/h/m/' )
The read
command blocks until something writes to named pipe, which does't happen until tee
completes.
(Note that this may not completely solve your problem, as in addition to the process synchronization you may need to contend with the fact that you have multiple processes writing to the same output file, and you don't have complete control over how the various writes are multiplexed (due to buffering, etc.)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With