python logging print traceback only in debug

Question

I am currently loading the python logger like this:

import logging
logging.basicConfig(level=logging.INFO)
log = logging.getLogger("myprogram")

and using it e. g. like this:

[...]
except FileNotFoundError:
    log.exception("could not open configuration file")
    sys.exit(1)

However, this will always print the traceback along with the error message:

ERROR:myprogram:could not open configuration file
Traceback (most recent call last):
[...]
FileNotFoundError: [Errno 2] No such file or directory: 
'not/existing/file.yml'

I do not want the traceback in the normal error output. Instead it should only print my error message and the exception info ("No such file...").

What is the recommended way of showing the traceback only when the loglevel is set to logging.DEBUG?

Answer 1

Log the exception at DEBUG level instead and set exc_info=True. logger.exception() is essentially a logger.error(..., exc_info=True) call, but you can log exception tracebacks at any level:

log.debug("could not open configuration file", exc_info=True)

It's the exc_info option that's important; from the documentation:

If exc_info does not evaluate as false, it causes exception information to be added to the logging message. If an exception tuple (in the format returned by sys.exc_info()) or an exception instance is provided, it is used; otherwise, sys.exc_info() is called to get the exception information.

You perhaps want to use printing (to stdout or stderr) to communicate with the end-user:

except FileNotFoundError as e:
    log.debug("could not open configuration file", exc_info=True)
    print("Could not open configuration file:", e.strerror, file=sys.stderr)
    sys.exit(1)

I included the system error message in the print output without the FileNotFoundError(...) representation.

If you use a command-line argument parser like argparse or click, then do use their user feedback API (which usually includes exiting too).

You can make the logging module produce user-level messages too, but if you want a single logger call to produce debug-friendly tracebacks in a file and user-friendly output on a console, you'd have to configure separate handlers for these use-cases with the console handler using a custom Formatter() class to override the formatException() method to alter how exceptions are shown. It's just much easier and clearer to separate logging and end-user communication.

Answer 2

I'd use a combination of exc_info and .getEffectiveLevel:

try:
    ...
except FileNotFoundError as ex:
   logger.error(ex, exc_info=log.getEffectiveLevel() == logging.DEBUG)

This way, the exception itself (FileNotFoundError) is always logged, but the stacktrace will only be logged if log level is debug.