How to completely reset Python stdlib logging module in an ipython session?

Question

I'd like to make repeated calls to Python scripts using %run in an ipython session, and for each of those scripts to log based on cmdline arguments passed via %run.

For example while debugging cmd.py I might over time want to run:

%run cmd.py
... logs with default behavior, e.g. to STDERR with root level WARN

%run cmd.py --log_level DEBUG --log_file /tmp/cmd.out
... logs with root level DEBUG to a file

%run cmd.py --log_level ERROR

Unfortunately this is difficult because the logging state created by logging.basicConfig persists after the first %run command (as is more generally true of all modules, and often desirable when using %run).

I realize that in full generality a series of %run commands like above will not be the same as running each command in a new process. However, it would be very convenient if things like the log_level and log_file could be re-initialized.

I've tried something like this in cmd.py:

import logging_config  # parse logging config from sys.argv
reload(logging_config) # re-parse cmdline if using %run multiple times

and logging_config.py does (condensed):

if logging_initialized:
    logging.getLogger().setLevel(lvl)
else:
    logging.basicConfig(level=lvl)
    logging_initialized = True

It works for simple cases but not if cmd.py imports libraries that also use logging. I've also experimented with logging.shutdown() (called at conclusion of each cmd.py) but that does not seem to help.

Answer 1

Don't use basicConfig() for this style of usage - it's meant for simple one-off configuration and, as the documentation says, subsequent calls after the first have no effect (it only does anything if the root logger has no handlers). This will be fine when your script is run from the shell prompt, but not from the interactive interpreter, IPython, IDLE, PythonWin, or other similar environment where the process you're interacting with doesn't exit.

Instead, use programmatic configuration, or fileConfig() or dictConfig() - these have modes where the completely replace the existing logging configuration with a new one.