Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

python logging with multiple modules does not work

Tags:

python

logging

I created some Python files keeping my functions a bit separated to ease working / fixing. All files are in one directory. The structure may get broken down to something like:

  • a.py (a class A with basic stuff)
  • b.py (a class B with basic stuff)
  • modA.py (create a class C deriving from A and B)
  • modB.py (create a class D deriving from A and B)
  • ...
  • main_a.py (using class C)
  • main_b.py (using class D)

Every module uses the logging stuff from python. An why so ever - only the root logger messages are written. And I don't see my error.

Here is a minimal example.

a.py

import logging
logger = logging.getLogger(__name__)

class A(object):
    def __init__(self):
        logger.debug("Instance of A")

b.py

import logging
logger = logging.getLogger(__name__)

class B(object):
    def __init__(self):
        logger.debug("Instance of B")

ab.py

import a
import b
import logging
logger = logging.getLogger(__name__)

class AB(a.A, b.B):
    def __init__(self):
        logger.debug("Instance of AB")
        a.A.__init__(self)
        b.B.__init__(self)

main_one.py

import sys
import ab

import logging
import logging.handlers
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
handler = logging.StreamHandler(stream=sys.stderr)
handler.setLevel(logging.DEBUG)
handler.setFormatter(logging.Formatter('%(name)s: %(message)s'))
logger.addHandler(handler)

logger.warning("The trouble starts")

ab = ab.AB()

I also tried to use something like self.logger = logging.getLogger(type(self).__name__) to do logging on a per class base, but the result is the same. So may one of you point out where I went wrong when reading the python logging manuals?

TIA.

EDIT 1: My solution

Thanks to both, @falsetru and @Jakub M., using both answers leads to a working solution.

First I put everything in a hierarchy.

main_one.py
  lib/
    __init__.py
    ab.py
    basic/
      __init__.py
      a.py
      b.py

Second I changed the logger = logging.getLogger(__name__) in main_one.py to logger = logging.getLogger() (No name for the root logger!).

That did the trick.

Very helpful was a code snippet on GitHub.

like image 765
Daniel Avatar asked Jun 27 '13 07:06

Daniel


People also ask

How do I create a multiple logging level in Python?

You can set a different logging level for each logging handler but it seems you will have to set the logger's level to the "lowest". In the example below I set the logger to DEBUG, the stream handler to INFO and the TimedRotatingFileHandler to DEBUG. So the file has DEBUG entries and the stream outputs only INFO.

Does Python logging module use Log4j?

If you've written Java code before, you might be familiar with the Log4j Java library. The Python logging module is largely and primarily based on that library.

Does Python logging block?

Developers use Python logging to file (a log file created by the Python logging module and filled with logging information via a handler) to collect this data. It is important that the log file works asynchronously. Otherwise, logging in Python may block the execution of the script.


1 Answers

Do print __name__ for each of your modules and see what you get there actually. You should put your modules into proper directories so __name__ is a "period separated hierarchical value". For example, if your file hierarchy looked like:

main_a.py
libs/
  __init__.py
  a.py
  modA.py

then __name__ of your modules (a.py and modA.py) would be libs.a and libs.modA

The name is potentially a period-separated hierarchical value, like foo.bar.baz (though it could also be just plain foo, for example). Loggers that are further down in the hierarchical list are children of loggers higher up in the list. For example, given a logger with a name of foo, loggers with names of foo.bar, foo.bar.baz, and foo.bam are all descendants of foo. The logger name hierarchy is analogous to the Python package hierarchy, and identical to it if you organise your loggers on a per-module basis using the recommended construction logging.getLogger(name). That’s because in a module, name is the module’s name in the Python package namespace.

like image 94
Jakub M. Avatar answered Sep 23 '22 07:09

Jakub M.