logging in multiple classes with module name in log

Question

I want to use the logging module instead of printing for debug information and documentation. The goal is to print on the console with DEBUG level and log to a file with INFO level.

I read through a lot of documentation, the cookbook and other tutorials on the logging module but couldn't figure out, how I can use it the way I want it. (I'm on python25)

I want to have the names of the modules in which the logs are written in my logfile.

The documentation says I should use logger = logging.getLogger(__name__) but how do I declare the loggers used in classes in other modules / packages, so they use the same handlers like the main logger? To recognize the 'parent' I can use logger = logging.getLogger(parent.child) but where do I know, who has called the class/method?`

The example below shows my problem, if I run this, the output will only have the __main__ logs in and ignore the logs in Class

This is my Mainfile:

# main.py

import logging
from module import Class

logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)

# create file handler which logs info messages
fh = logging.FileHandler('foo.log', 'w', 'utf-8')
fh.setLevel(logging.INFO)

# create console handler with a debug log level
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)

# creating a formatter
formatter = logging.Formatter('- %(name)s - %(levelname)-8s: %(message)s')

# setting handler format
fh.setFormatter(formatter)
ch.setFormatter(formatter)

# add the handlers to the logger
logger.addHandler(fh)
logger.addHandler(ch)

if __name__ == '__main__':
    logger.info('Script starts')
    logger.info('calling class Class')
    c = Class()
    logger.info('calling c.do_something()')
    c.do_something()
    logger.info('calling c.try_something()')
    c.try_something()

Module:

# module.py

imnport logging

class Class:
    def __init__(self):
        self.logger = logging.getLogger(__name__) # What do I have to enter here?
        self.logger.info('creating an instance of Class')
        self.dict = {'a':'A'}
    def do_something(self):
        self.logger.debug('doing something')
        a = 1 + 1
        self.logger.debug('done doing something')
    def try_something(self):
        try:
            logging.debug(self.dict['b'])
        except KeyError, e:
            logging.exception(e)

Output in console:

- __main__ - INFO    : Script starts
- __main__ - INFO    : calling class Class
- __main__ - INFO    : calling c.do_something()
- __main__ - INFO    : calling c.try_something()
No handlers could be found for logger "module"

Besides: Is there a way to get the module names were the logs ocurred in my logfile, without declaring a new logger in each class like above? Also like this way I have to go for self.logger.info() each time I want to log something. I would prefer to use logging.info() or logger.info() in my whole code.

Is a global logger perhaps the right answer for this? But then I won't get the modules where the errors occur in the logs...

And my last question: Is this pythonic? Or is there a better recommendation to do such things right.

Answer 1

In your main module, you're configuring the logger of name '__main__' (or whatever __name__ equates to in your case) while in module.py you're using a different logger. You either need to configure loggers per module, or you can configure the root logger (by configuring logging.getLogger()) in your main module which will apply by default to all loggers in your project.

I recommend using configuration files for configuring loggers. This link should give you a good idea of good practices: http://victorlin.me/posts/2012/08/26/good-logging-practice-in-python

EDIT: use %(module) in your formatter to include the module name in the log message.

Answer 2

The generally recommended logging setup is having at most 1 logger per module.

If your project is properly packaged, __name__ will have the value of "mypackage.mymodule", except in your main file, where it has the value "__main__"

If you want more context about the code that is logging messages, note that you can set your formatter with a formatter string like %(funcName)s, which will add the function name to all messages.

If you really want per-class loggers, you can do something like:

class MyClass:
    def __init__(self):
        self.logger = logging.getLogger(__name__+"."+self.__class__.__name__)