python loggers as children of __main__

Question

I'm having trouble getting child loggers named properly in python (2.7). I have the following file structure:

-mypackage
  -__init__.py
  -main.py
  -log
    -__init__.py
    -logfile.log
  -src
    -__init__.py
    -logger.py
    -otherfile.py

The contents of main.py are:

import logging
import src.logger
from src.otherfile import Foo

logger = logging.getLogger(__name__)
logger.info('Logging from main')
foo = Foo()

The contents of otherfile.py are:

import logging
class Foo():
    def __init__(self):
        self.logger = logging.getLogger(__name__)
        self.logger.info('Logging from class in otherfile')

The contents of logger.py are:

import os
import logging
logdir = os.path.dirname(__file__)
logfile = os.path.join(logdir, '../log/controller.log')
logger = logging.getLogger('__main__')
logger.setLevel(logging.DEBUG)

fh = logging.FileHandler(logfile)
fh.setLevel(logging.DEBUG)

formatter = logging.Formatter('%(asctime)s - $(name)s - %(levelname)s: %(message)s')
fh.setFormatter(formatter)

logger.addHandler(fh)

logger.info('logging from logger.py')

I used logging.getLogger(__name__) in each file based on the docs. The exception is logger.py, where I name the logger __main__ so that it would run from top down and not try to derive everything from a buried file.

When I run main.py, it logs correctly from logger.py and main.py, but the logger from otherfile.py isn't derived correctly from the main logger.

How do I get the logger in otherfile.py to derive from my main logger?

Answer 1

In logger.py you are configuring the "__main__" logger. I was tricked by the fact that in main.py you use __name__. Since you are invoking python main.py, __name__ evaluates to "__main__". Right.

This can become a problem since when imported (instead of executed), main.py's logger won't be "__main__" but "main". It can be fixed by making your package executable: rename main.py to __main__.py and running your package like this:

python -m mypackage

This way, logger names (actually module __name__'s) will remain consistent.

That said, in no way the logger that you configure in logger.py is a parent of the logger in otherfile.py. The real parent of that logger is called "mypackage" but you haven't configured it, so it's logs are invisible.

You have several choices, you can configure (set log level, handler and formatter):

• the root logger : logger = logging.getLogger()
• mypackage's logger : logger = logger.getLogger(__name__) in mypackage.__init__
• ... or go down to the level of granularity you wish.

Answer 2

You can easily create hierarchies of loggers by calling separating levels of loggers by a dot ("."). For example, the logger returned by calling logging.getLogger('__main__.' + __name__) inherits all properties from the logger returned by logging.getLogger('__main__'). This behaviour is described here: https://docs.python.org/2/library/logging.html#module-level-functions.