When to use sys.stdout instead of sys.stderr?

Question

My boss told me to look at some old code where everything is being sent to stderr. I know that stderr should have the warnings and errors, but when should they really should go to stdout instead?

This program is a service. Some of the messages that it sends to stderr are:

if pid is None:
    message = "pidfile {0} is not running. \n"
    sys.stderr.write(message.format(self.pidfile))

So this one is an output, right? A message about being successful or not should go to stdout, because it's not an error?

And then I got another one that is an exception:

except OSError as err:
    sys.stderr.write('fork #1 failed: {0}\n'.format(err))
    sys.exit(1)

This one as an exception, so it should go to stderr, because it's an error? Or should it go to stdout, because it is a output message just saying that the program failed?

I know what stdout and stderr are --- I read quite a bit before asking. My problem here is being able to identify which messages should go to stderr --- just fatal errors, or any kind of error message?

Both the examples are error messages, but one is just a "Program is not running". I'm having trouble choosing where that kind of message should be sent.

Answer 1

Fatal errors should certainly go to stderr. Normal output should go to stdout.

There is a gray area for informative messages, but in general they should not go to stdout if they would render the output unusable. People will often direct stdout to a file and stderr to either the terminal or a log file.

You need to decide what division makes the most sense.

Answer 2

Tom Karzes has already given you a correct answer, but to state a more extreme position...

I find it helps to think of stdout and stderr as "standard output" and "standard not-output". You said you thought a message "should go to stdout because it's not an error", but that's backward: stdout is the stream with the high barrier to entry, not stderr.

A message should go to stderr (or a log file, or anything but stdout) unless it is provably part of the desired output --- the final product that the user actually wants. If you can't justify its inclusion with a sentence that starts, "The user asked for...", don't pollute stdout with it. (The next process in the pipeline will thank you.)

Odds are, your users didn't run your program because they wanted to watch a twirling baton, or to read a dozen "Attempting to reconnect..." messages, or even to learn which forks had failed, so those should not be on stdout.

Finally, some messages shouldn't go to either output stream. stderr is presumably being read by a user, so try to limit its output to things that user cares about right now. Keep it free of "situation normal" messages, unless the user has specifically requested them by turning up the verbosity (usually --verbose or --debug). At least allow the user to silence non-errors with a --quiet or --silent option. Most "informational" messages should be sent to a log file instead of the console.