How to configure Logging in Python

Question

I am new to Python and just starting one project. I am used to use log4j in Java and I would like to log all modules and classes in Python as I do in Java.

In Java I have one log configuration file in src folder named log4j.properties like below:

log4j.rootLogger=DEBUG, Console, fileout  log4j.appender.Console=org.apache.log4j.ConsoleAppender log4j.appender.Console.layout=org.apache.log4j.PatternLayout log4j.appender.Console.layout.ConversionPattern=%d{dd/MM/yyyy HH:mm:ss} %5p [%t] (%F:%L) - %m%n  log4j.appender.fileout=org.apache.log4j.RollingFileAppender log4j.appender.fileout.File=servidor.log log4j.appender.fileout.layout=org.apache.log4j.PatternLayout log4j.appender.fileout.layout.ConversionPattern=%d{dd/MM/yyyy HH:mm:ss} (%F:%L) %p %t %c - %m%n 

It logs to console and a file.

In my classes, I only have to import log4j and add a static attribute to recovery the log4j logger with config loaded then all classes will be logging in console and file. The configuration file is loaded automaticaly by the name. For example:

import org.apache.log4j.Logger;  public class Main {     public static Logger logger = Logger.getLogger(Main.class);     public static void main(String[] args) {         logger.info("Hello");     } } 

Now I am having problem to setup logging in Python, I have read the docs but I couldn't find a way to use it in many modules/classes. How could I setup Python logging in an easy way to log my modules and classes without code much in every module/class? Is it possible to reproduce same code that I have wrote in Python?

Answer 1

Actually in Python it looks pretty much similar. There are different ways to do it. I usually create a logger class which is very simple:

import os import logging  import settings   # alternativly from whereever import settings    class Logger(object):      def __init__(self, name):         name = name.replace('.log','')         logger = logging.getLogger('log_namespace.%s' % name)    # log_namespace can be replaced with your namespace          logger.setLevel(logging.DEBUG)         if not logger.handlers:             file_name = os.path.join(settings.LOGGING_DIR, '%s.log' % name)    # usually I keep the LOGGING_DIR defined in some global settings file             handler = logging.FileHandler(file_name)             formatter = logging.Formatter('%(asctime)s %(levelname)s:%(name)s %(message)s')             handler.setFormatter(formatter)             handler.setLevel(logging.DEBUG)             logger.addHandler(handler)         self._logger = logger      def get(self):         return self._logger 

Then if I want to log something in a class or module I simply import the logger and create an instance. Passing the class name will create one file for each class. The logger can then log messages to its file via debug, info, error, etc.:

from module_where_logger_is_defined import Logger  class MyCustomClass(object):      def __init__(self):         self.logger = Logger(self.__class__.__name__).get()   # accessing the "private" variables for each class      def do_something():         ...         self.logger.info('Hello')      def raise_error():         ...         self.logger.error('some error message') 

Updated answer

Over the years I changed how I am using Python logging quite a bit. Mostly based in good practices I configure the logging of the whole application once in whatever module is loaded first during startup of the application and then use individual loggers in each file. Example:

 # app.py (runs when application starts)  import logging import os.path  def main():     logging_config = {         'version': 1,         'disable_existing_loggers': False,         'formatters': {             'standard': {                 'format': '%(asctime)s [%(levelname)s] %(name)s: %(message)s'             },         },         'handlers': {             'default_handler': {                 'class': 'logging.FileHandler',                 'level': 'DEBUG',                 'formatter': 'standard',                 'filename': os.path.join('logs', 'application.log'),                 'encoding': 'utf8'             },         },         'loggers': {             '': {                 'handlers': ['default_handler'],                 'level': 'DEBUG',                 'propagate': False             }         }     }     logging.config.dictConfig(logging_config)     # start application ...  if __name__ == '__main__':     main() 

 # submodule.py (any application module used later in the application)  import logging  # define top level module logger logger = logging.getLogger(__name__)  def do_something():     # application code ...     logger.info('Something happended')     # more code ...     try:         # something which might break     except SomeError:         logger.exception('Something broke')         # handle exception     # more code ... 

The above is the recommended way of doing this. Each module defines its own logger and can easily identify based on the __name__ attribute which message was logged in which module when you inspect the logs. This removes the boilerplate from my original answer and instead uses the logging.config module from the Python standard library.

Answer 2

The docs provide a pretty good example of using your logger in multiple modules. Basically, you set up the logging once at the start of your program. Then, you import the logging module wherever you want to have logging, and use it.

myapp.py

import logging import mylib  def main():     logging.basicConfig(filename='myapp.log', level=logging.INFO)     logging.info('Started')     mylib.do_something()     logging.info('Finished')  if __name__ == '__main__':     main() 

mylib.py

import logging  def do_something():     logging.info('Doing something') 

This example shows a very simplistic logger setup, but you could very easily use the various ways to configure logging to set up more advanced scenarios.