How to enable debug mode as an option using Python's logging module

Question

Let's say I pass in a parameter of --debug to my script where I want it to display additional text when it otherwise doesn't display it by default.

Going to keep it really simple here. How do I get logger.debug to display/log? Is it just a matter of "enabling" it if --debug is passed?

import logging
logger = logging.getLogger()
formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')

fh = logging.FileHandler('xyz.log')
fh.setLevel(logging.INFO)
fh.setFormatter(formatter)
logger.addHandler(fh)

logger.error("This is an error")
logger.info("Info 123")
logger.debug("FYI, here's what you don't see unless you enable me.")

Maybe?

fh.setLevel(logging.DEBUG)
if option == 'debug'
    logger.debug("FYI, here's what you don't see unless you enable me.")

Answer 1

Python logging system is quite complex, deserving a question in an interview.

logger = logging.getLogger(__name__)
logger.debug("FYI, here's what you don't see unless you enable me.")

To see the message in, let's say, console you have to:

1. Enable logger of the current module (see __name__ in the code) or some of the parent loggers (for example the root logger) for level DEBUG.
2. Attach a handler for stdout to logger of the current module or some of its parents with level DEBUG.

Using dictConfig is easier to set up the logging configuration:

import logging.config


LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'simple': {
            'format': '%(levelname)s: %(message)s'
        },
    },
    'handlers': {
        'console': {
            'level': 'DEBUG',
            'class': 'logging.StreamHandler',
        },
    },
    'loggers': {
        '': {
            'level': 'DEBUG',
            'handlers': ['console'],
        },
    },
}


logging.config.dictConfig(LOGGING)

logging.getLogger(__name__).debug('This is a debug message')

In the example above you will see the message in the console.

But if you change level of the root logger or of the console handler to INFO -- you will not see the message.

You keep the logging configuration in a separate, environment-dependent file.

See also a very useful package when fiddling with logging configuration: http://rhodesmill.org/brandon/2012/logging_tree/

Answer 2

fh = logging.FileHandler('xyz.log')
fh.setLevel(logging.DEBUG if option == 'debug' else logging.INFO)    
fh.setFormatter(formatter)
logger.addHandler(fh)

To handle arguments which are passed to your script you can use one of bunch libs such as argparse or docopt.

Answer 3

It's displaying logging logger.info since it's what you set it to do with fh.setLevel(logging.INFO). Use fh.setLevel(logging.DEBUG) to register logger.debug calls or both depending on option. You can also have multiple Filehandlers if you want to keep INFO and DEBUG seperate. Remember that your log is stored in xyz.log - if you want to stop logging during runtime call logger.disabled = True.