Questions about FIFOs and file descriptor

Question

I write a script like this:

N=5
FIFO=/tmp/$$.fifo
mkfifo $FIFO

loop(){
    for i in $(seq 1 $N); do
        read tmp < $FIFO
        echo "$i out"
    done
}

loop &
LOOP_PID=$!
for i in $(seq 1 $N); do
    echo $i > $FIFO
    echo "$i in"
done
wait $LOOP_PID

When I run the script, It stop at wait $LOOP_PID and could not continue.

So I modify the script with file descriptor:

N=5
FIFO=/tmp/$$.fifo
mkfifo $FIFO
exec 3<>$FIFO

loop(){
    for i in $(seq 1 $N); do
        read -u3 tmp
        echo "$i out"
    done
}

loop &
LOOP_PID=$!
for i in $(seq 1 $N); do
    echo $i >&3
    echo "$i in"
done
wait $LOOP_PID

It was ok.

When I use FIFOs directly, it could not read data from FIFOs continuously and it would hang on. When I use file descriptor, it was ok. what is the reason?

Answer 1

Replace this:

loop(){
    for i in $(seq 1 $N); do
        read tmp < $FIFO
        echo "$i out"
    done
}

With this:

loop(){
    for i in $(seq 1 $N); do
        read tmp
        echo "$i out"
    done < $FIFO
}

This keeps the fifo open, rather than re-opening and re-closing it with every loop.

FIFOs are quite tricky:

1. An attempt to write to a FIFO will block unless another process is ready to read from the FIFO.

2. If the process reading from the FIFO closes the FIFO, any unread information is lost.

This means that how your scripts above behave may depend on accidents of timing. When read tmp < $FIFO was executing, how many lines had been written to the FIFO? read will read only the first and, when the FIFO is closed, the rest will be discarded.

How the exec statement helps

Let's compare two scripts. The first uses the FIFO directly:

#!/bin/sh
fifo=/tmp/$$.myfifo
mkfifo "$fifo"
echo $'1\n2\n3\n4'>"$fifo"

for i in {1..4}
do
    read  tmp
    echo $tmp
done <"$fifo"

The above script will hang during the first echo while it waits for a process to start to read the FIFO. Because it hangs there, the read tmp statement is never reached and this script produces no output.

The second uses exec to create a file handle:

#!/bin/sh
fifo=/tmp/$$.myfifo
mkfifo "$fifo"
exec 3<>"$fifo"
echo $'1\n2\n3\n4'>&3

for i in {1..4}
do
    read -u3 tmp
    echo $tmp
done

This script does not hang and will produce four lines of output. The difference is that the shell provides buffering on the file handle. So, when the first echo statement tries to write to the FIFO, the shell is ready to read from the FIFO. The data that the shell has read is kept available for the read -u3 tmp statement.