Per-file/module logger in Python

Question

I have some Python code I need to add logging to.

I've always preferred nice big C macro looking statements like "DEBUG()", "ERROR()", etc for logging. I feel it makes the code easier to read (no objects) when trace-points are visually differentiable from the actual code.

I also would like to be able to set logging levels at a per-module level.

How could I make a module called "log" that is capable of doing this (while making use of the Python std library logging module)?

E.g.:

File: main.py

# This imports LOG_MODULE_NAME, DEBUG, WARN, etc
from log import *
import my_module

LOG_MODULE_NAME("main")

log.set_level("main", log.LVL_DEBUG)
log.set_level("my_module", log.LVL_WARN)

if __name__ == "__main__":
    foo = my_module.myFunc(2)

DEBUG("Exiting main.py")

File: my_module.py

from log import *

LOG_MODULE_NAME("my_module")


def myFunc(x):
    DEBUG("Entering function")
    if x != 1:
         WARN("I thought it would be 1")
    DEBUG("Exiting function")
    return x+1

I'd expect output to look something like:

[WARN:my_module - my_module.py:9] I thought it would be 1
[DEBUG:main - main.py:11] Exiting main.py

Answer 1

These answers seem to skip the very simple idea that functions are first-class objects which allows for:

def a_function(n):
    pass

MY_HAPPY_NAME = a_function
MY_HAPPY_NAME(15) # which is equivalent to a_function(15)

Except I recommend that you don't do this. PEP8 coding conventions are widely used because life is too short to make me have to read identifiers in the COBOLy way that you like to write them.

Some other answers also use the global statement which is almost always not needed in Python. If you are making module-level logger instances then

import logging

log = logging.getLogger(__name__)

def some_method():
    ...
    log.debug(...)

is a perfectly workable, concise way of using the module-level variable log. You could even do

log = logging.getLogger(__name__)
DEBUG = log.debug

def some_method():
    ...
    DEBUG(...)

but I'd reserve the right to call that ugly.

Answer 2

To expand on the @2rs2ts notion, you can just log by module too as shown in the Python logging HOWTO.

A good convention to use when naming loggers is to use a module-level logger, in each module which uses logging, named as follows:

logger = logging.getLogger(__name__) 

This means that logger names track the package/module hierarchy, and it’s intuitively obvious where events are logged just from the logger name

Make the first 2 lines of all your files this:

import logging
logger = logging.getLogger(__name__) 

Then you have per-module logging that is easily customizable.

You can log to different levels via:

logger.debug("I just saw this %s" , var )
logger.info("I just saw this %s" , var )
logger.warn("I just saw this %s" , var )

Note that I passed the string templating variable as an *arg, and not a % to immediately template the string. If your logger is set to be higher than the level you log at, you save yourself the CPU cycles. There are a handful of tricks like that in the logging docs.

You can set the logging level and where to log on whatever app/script you run; the logging module will handle the rest.