How do I copy stderr without stopping it writing to the terminal?

Question

I want to write a shell script that runs a command, writing its stderr to my terminal as it arrives. However, I also want to save stderr to a variable, so I can inspect it later.

How can I achieve this? Should I use tee, or a subshell, or something else?

I've tried this:

# Create FD 3 that can be used so stdout still comes through 
exec 3>&1

# Run the command, piping stdout to normal stdout, but saving stderr.
{ ERROR=$( $@ 2>&1 1>&3) ; }

echo "copy of stderr: $ERROR"

However, this doesn't write stderr to the console, it only saves it.

I've also tried:

{ $@; } 2> >(tee stderr.txt >&2 )

echo "stderr was:"
cat stderr.txt

However, I don't want the temporary file.

Answer 1

Credit goes to @Etan Reisner for the fundamentals of the approach; however, it's better to use tee with /dev/stderr rather than /dev/tty in order to preserve normal behavior (if you send to /dev/tty, the outside world doesn't see it as stderr output, and can neither capture nor suppress it):

Here's the full idiom:

exec 3>&1   # Save original stdout in temp. fd #3.
# Redirect stderr to *captured* stdout, send stdout to *saved* stdout, also send
# captured stdout (and thus stderr) to original stderr.
errOutput=$("$@" 2>&1 1>&3 | tee /dev/stderr)
exec 3>&-   # Close temp. fd.

echo "copy of stderr: $errOutput"    

Answer 2

I often want to do this, and find myself reaching for /dev/stderr, but there can be problems with this approach; for example, Nix build scripts give "permission denied" errors if they try to write to /dev/stdout or /dev/stderr.

After reinventing this wheel a few times, my current approach is to use process substitution as follows:

myCmd 2> >(tee >(cat 1>&2))

Reading this from the outside in:

This will run myCmd, leaving its stdout as-is. The 2> will redirect the stderr of myCmd to a different destination; the destination here is >(tee >(cat 1>&2)) which will cause it to be piped into the command tee >(cat 1>&2).

The tee command duplicates its input (in this case, the stderr of myCmd) to its stdout and to the given destination. The destination here is >(cat 1>&2), which will cause the data to be piped into the command cat 1>&2.

The cat command just passes its input straight to stdout. The 1>&2 redirects stdout to go to stderr.

Reading from the inside out:

The cat 1>&2 command redirects its stdin to stderr, so >(cat 1>&2) acts like /dev/stderr.

Hence tee >(cat 1>&2) duplicates its stdin to both stdout and stderr, acting like tee /dev/stderr.

We use 2> >(tee >(cat 1>&2)) to get 2 copies of stderr: one on stdout and one on stderr.

We can use the copy on stdout as normal, for example storing it in a variable. We can leave the copy on stderr to get printed to the terminal.

We can combine this with other redirections if we like, e.g.

# Create FD 3 that can be used so stdout still comes through 
exec 3>&1

# Run the command, redirecting its stdout to the shell's stdout,
# duplicating its stderr and sending one copy to the shell's stderr
# and using the other to replace the command's stdout, which we then
# capture
{ ERROR=$( $@ 2> >(tee >(cat 1>&2)) 1>&3) ; }

echo "copy of stderr: $ERROR"