How do I get both STDOUT and STDERR to go to the terminal and a log file?

Question

Use "tee" to redirect to a file and the screen. Depending on the shell you use, you first have to redirect stderr to stdout using

./a.out 2>&1 | tee output

or

./a.out |& tee output

In csh, there is a built-in command called "script" that will capture everything that goes to the screen to a file. You start it by typing "script", then doing whatever it is you want to capture, then hit control-D to close the script file. I don't know of an equivalent for sh/bash/ksh.

Also, since you have indicated that these are your own sh scripts that you can modify, you can do the redirection internally by surrounding the whole script with braces or brackets, like

  #!/bin/sh
  {
    ... whatever you had in your script before
  } 2>&1 | tee output.file

Approaching half a decade later...

I believe this is the "perfect solution" sought by the OP.

Here's a one liner you can add to the top of your Bash script:

exec > >(tee -a $HOME/logfile) 2>&1

Here's a small script demonstrating its use:

#!/usr/bin/env bash

exec > >(tee -a $HOME/logfile) 2>&1

# Test redirection of STDOUT
echo test_stdout

# Test redirection of STDERR
ls test_stderr___this_file_does_not_exist

(Note: This only works with Bash. It will not work with /bin/sh.)

Adapted from here; the original did not, from what I can tell, catch STDERR in the logfile. Fixed with a note from here.

The Pattern

the_cmd 1> >(tee stdout.txt ) 2> >(tee stderr.txt >&2 )

This redirects both stdout and stderr separately, and it sends separate copies of stdout and stderr to the caller (which might be your terminal).

• In zsh, it will not proceed to the next statement until the tees have finished.

• In bash, you may find that the final few lines of output appear after whatever statement comes next.

In either case, the right bits go to the right places.

---

Explanation

Here's a script (stored in ./example):

#! /usr/bin/env bash
the_cmd()
{
    echo out;
    1>&2 echo err;
}

the_cmd 1> >(tee stdout.txt ) 2> >(tee stderr.txt >&2 )

Here's a session:

$ foo=$(./example)
    err

$ echo $foo
    out

$ cat stdout.txt
    out

$ cat stderr.txt
    err

Here's how it works:

1. Both tee processes are started, their stdins are assigned to file descriptors. Because they're enclosed in process substitutions, the paths to those file descriptors are substituted in the calling command, so now it looks something like this:

the_cmd 1> /proc/self/fd/13 2> /proc/self/fd/14

2. the_cmd runs, writing stdout to the first file descriptor, and stderr to the second one.

3. In the bash case, once the_cmd finishes, the following statement happens immediately (if your terminal is the caller, then you will see your prompt appear).

4. In the zsh case, once the_cmd finishes, the shell waits for both of the tee processes to finish before moving on. More on this here.

5. The first tee process, which is reading from the_cmd's stdout, writes a copy of that stdout back to the caller because that's what tee does. Its outputs are not redirected, so they make it back to the caller unchanged

6. The second tee process has it's stdout redirected to the caller's stderr (which is good, because it's stdin is reading from the_cmd's stderr). So when it writes to its stdout, those bits go to the caller's stderr.

This keeps stderr separate from stdout both in the files and in the command's output.

If the first tee writes any errors, they'll show up in both the stderr file and in the command's stderr, if the second tee writes any errors, they'll only show up only in the terminal's stderr.

the to redirect stderr to stdout append this at your command: 2>&1 For outputting to terminal and logging into file you should use tee

Both together would look like this:

 mycommand 2>&1 | tee mylogfile.log

EDIT: For embedding into your script you would do the same. So your script

#!/bin/sh
whatever1
whatever2
...
whatever3

would end up as

#!/bin/sh
( whatever1
whatever2
...
whatever3 ) 2>&1 | tee mylogfile.log

EDIT: I see I got derailed and ended up answering a different question from the one asked. The answer to the real question is at the bottom of Paul Tomblin's answer. (If you want to enhance that solution to redirect stdout and stderr separately for some reason, you could use the technique I describe here.)

---

I've been wanting an answer that preserves the distinction between stdout and stderr. Unfortunately all of the answers given so far that preserve that distinction are race-prone: they risk programs seeing incomplete input, as I pointed out in comments.

I think I finally found an answer that preserves the distinction, is not race prone, and isn't terribly fiddly either.

First building block: to swap stdout and stderr:

my_command 3>&1 1>&2 2>&3-

Second building block: if we wanted to filter (e.g. tee) only stderr, we could accomplish that by swapping stdout&stderr, filtering, and then swapping back:

{ my_command 3>&1 1>&2 2>&3- | stderr_filter;} 3>&1 1>&2 2>&3-

Now the rest is easy: we can add a stdout filter, either at the beginning:

{ { my_command | stdout_filter;} 3>&1 1>&2 2>&3- | stderr_filter;} 3>&1 1>&2 2>&3-

or at the end:

{ my_command 3>&1 1>&2 2>&3- | stderr_filter;} 3>&1 1>&2 2>&3- | stdout_filter

To convince myself that both of the above commands work, I used the following:

alias my_command='{ echo "to stdout"; echo "to stderr" >&2;}'
alias stdout_filter='{ sleep 1; sed -u "s/^/teed stdout: /" | tee stdout.txt;}'
alias stderr_filter='{ sleep 2; sed -u "s/^/teed stderr: /" | tee stderr.txt;}'

Output is:

...(1 second pause)...
teed stdout: to stdout
...(another 1 second pause)...
teed stderr: to stderr

and my prompt comes back immediately after the "teed stderr: to stderr", as expected.

Footnote about zsh:

The above solution works in bash (and maybe some other shells, I'm not sure), but it doesn't work in zsh. There are two reasons it fails in zsh:

1. the syntax 2>&3- isn't understood by zsh; that has to be rewritten as 2>&3 3>&-
2. in zsh (unlike other shells), if you redirect a file descriptor that's already open, in some cases (I don't completely understand how it decides) it does a built-in tee-like behavior instead. To avoid this, you have to close each fd prior to redirecting it.

So, for example, my second solution has to be rewritten for zsh as {my_command 3>&1 1>&- 1>&2 2>&- 2>&3 3>&- | stderr_filter;} 3>&1 1>&- 1>&2 2>&- 2>&3 3>&- | stdout_filter (which works in bash too, but is awfully verbose).

On the other hand, you can take advantage of zsh's mysterious built-in implicit teeing to get a much shorter solution for zsh, which doesn't run tee at all:

my_command >&1 >stdout.txt 2>&2 2>stderr.txt

(I wouldn't have guessed from the docs I found that the >&1 and 2>&2 are the thing that trigger zsh's implicit teeing; I found that out by trial-and-error.)

Use the tee program and dup stderr to stdout.

 program 2>&1 | tee > logfile