Python logging configuration file

Question

I seem to be having some issues while attempting to implement logging into my python project.

I'm simply attempting to mimic the following configuration:

Python Logging to Multiple Destinations

However instead of doing this inside of code, I'd like to have it in a configuration file.

Below is my config file:

[loggers]
keys=root

[logger_root]
handlers=screen,file

[formatters]
keys=simple,complex

[formatter_simple]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s

[formatter_complex]
format=%(asctime)s - %(name)s - %(levelname)s - %(module)s : %(lineno)d - %(message)s

[handlers]
keys=file,screen

[handler_file]
class=handlers.TimedRotatingFileHandler
interval=midnight
backupCount=5
formatter=complex
level=DEBUG
args=('logs/testSuite.log',)

[handler_screen]
class=StreamHandler
formatter=simple
level=INFO
args=(sys.stdout,)

The problem is that my screen output looks like:
2010-12-14 11:39:04,066 - root - WARNING - 3
2010-12-14 11:39:04,066 - root - ERROR - 4
2010-12-14 11:39:04,066 - root - CRITICAL - 5

My file is output, but looks the same as above (although with the extra information included). However the debug and info levels are not output to either.

I am on Python 2.7

Here is my simple example showing failure:

import os
import sys
import logging
import logging.config

sys.path.append(os.path.realpath("shared/"))
sys.path.append(os.path.realpath("tests/"))

class Main(object):

  @staticmethod
  def main():
    logging.config.fileConfig("logging.conf")
    logging.debug("1")
    logging.info("2")
    logging.warn("3")
    logging.error("4")
    logging.critical("5")

if __name__ == "__main__":
  Main.main()

Answer 1

It looks like you've set the levels for your handlers, but not your logger. The logger's level filters every message before it can reach its handlers and the default is WARNING and above (as you can see). Setting the root logger's level to NOTSET as you have, as well as setting it to DEBUG (or whatever is the lowest level you wish to log) should solve your issue.

Answer 2

Adding the following line to the root logger took care of my problem:

level=NOTSET

Answer 3

Just add log level in [logger_root]. It is worked.

[logger_root]
level=DEBUG
handlers=screen,file

Answer 4

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import logging
import logging.handlers
from logging.config import dictConfig

logger = logging.getLogger(__name__)

DEFAULT_LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
}
def configure_logging(logfile_path):
    """
    Initialize logging defaults for Project.

    :param logfile_path: logfile used to the logfile
    :type logfile_path: string

    This function does:

    - Assign INFO and DEBUG level to logger file handler and console handler

    """
    dictConfig(DEFAULT_LOGGING)

    default_formatter = logging.Formatter(
        "[%(asctime)s] [%(levelname)s] [%(name)s] [%(funcName)s():%(lineno)s] [PID:%(process)d TID:%(thread)d] %(message)s",
        "%d/%m/%Y %H:%M:%S")

    file_handler = logging.handlers.RotatingFileHandler(logfile_path, maxBytes=10485760,backupCount=300, encoding='utf-8')
    file_handler.setLevel(logging.INFO)

    console_handler = logging.StreamHandler()
    console_handler.setLevel(logging.DEBUG)

    file_handler.setFormatter(default_formatter)
    console_handler.setFormatter(default_formatter)

    logging.root.setLevel(logging.DEBUG)
    logging.root.addHandler(file_handler)
    logging.root.addHandler(console_handler)



[31/10/2015 22:00:33] [DEBUG] [yourmodulename] [yourfunction_name():9] [PID:61314 TID:140735248744448] this is logger infomation from hello module

I think you should add the disable_existing_loggers to false.

Answer 5

A simple approach to both write to terminal and file would be as following:

import logging.config

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s [%(levelname)s] %(message)s",
    handlers=[
        logging.FileHandler("log_file.log"),
        logging.StreamHandler()
    ]
)
logger = logging.getLogger(__name__)

And then use it in your code like this:

logger.info('message')
logger.error('message')