Set up logging early: Catch warnings emmited during importing

Question

I want all warnings to be handled by the way I want logs to be handled. Even these emitted during importing libraries.

This means the configuration of the logging must be done before importing libraries.

After searching some time I think configuring logging in a custom sitecustomize.py could be a solution.

But sitecustomize.py is somehow black magic only few people know it, and even less people use it.

Is there a more obvious way to ensure that my logging config is done before libraries get imported?

Unfortunately there seems to be no way to configure logging via environment variables.

Update

I got no answer which is acceptable according to my point of view. I think you need to split logging into two areas of responsibility:

• Environment: set up logging: Which format, which files ...
• Code: use logging. Do emit info, warn and error messages.

As soon as the first line of code gets executed by the python interpreter, the area "environment" has given the responsibility to the code. Now it is too late to do any configuration. I would like to see setting up the logging to be a part of calling the python interpreter.

One solution could be an environment variable:

PYTHON_LOGGING_CONFIG=/path-to-env/etc/config.yaml

or an argument to the interpreter:

python --logging-config=path-to-env/etc/config.yaml script.py

Answer 1

There is apparently no way around explicitly configuring logging before importing anything else if you want logging to cover the imports.

However, if you want logging configuration to be more obvious, there are various tricks that will help you do that. For example, your main script could be as short as this, just to emphesize the special case:

import siteconfiguration
siteconfiguration.configure_site()  # This must be done before any other import

import application


if __name__ == "__main__":
    application.run()

That still breaks the rule of having all imports at the top, but it also clearly states why that is done. Everyone working with that code will understand your intentions.

Special cases aren't special enough to break the rules.

Although practicality beats purity.

(The Zen of Python, by Tim Peters)