Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Using logging in multiple modules

I have a small python project that has the following structure -

Project   -- pkg01    -- test01.py  -- pkg02    -- test02.py  -- logging.conf 

I plan to use the default logging module to print messages to stdout and a log file. To use the logging module, some initialization is required -

import logging.config  logging.config.fileConfig('logging.conf') logger = logging.getLogger('pyApp')  logger.info('testing') 

At present, I perform this initialization in every module before I start logging messages. Is it possible to perform this initialization only once in one place such that the same settings are reused by logging all over the project?

like image 650
Quest Monger Avatar asked Mar 31 '13 07:03

Quest Monger


People also ask

Can I have two loggers in Python?

if you transform the second module in a class, you can simply: declare the logger in the first modulue. create the new class passing the 2 logger as parameters. use the logger in the new class.

Can a logger have multiple handlers?

Python logger with multiple handlers (stream, file, etc.), so you can write log to multiple targets at once · GitHub.

Is Python logging blocking?

The built-in python logger is I/O blocking. This means that using the built-in logging module will interfere with your asynchronous application performance. aiologger aims to be the standard Asynchronous non-blocking logging for python and asyncio.

Is logging a module in Python?

Python comes with a logging module in the standard library that provides a flexible framework for emitting log messages from Python programs. This module is widely used by libraries and is the first go-to point for most developers when it comes to logging.


2 Answers

Best practice is, in each module, to have a logger defined like this:

import logging logger = logging.getLogger(__name__) 

near the top of the module, and then in other code in the module do e.g.

logger.debug('My message with %s', 'variable data') 

If you need to subdivide logging activity inside a module, use e.g.

loggerA = logging.getLogger(__name__ + '.A') loggerB = logging.getLogger(__name__ + '.B') 

and log to loggerA and loggerB as appropriate.

In your main program or programs, do e.g.:

def main():     "your program code"  if __name__ == '__main__':     import logging.config     logging.config.fileConfig('/path/to/logging.conf')     main() 

or

def main():     import logging.config     logging.config.fileConfig('/path/to/logging.conf')     # your program code  if __name__ == '__main__':     main() 

See here for logging from multiple modules, and here for logging configuration for code which will be used as a library module by other code.

Update: When calling fileConfig(), you may want to specify disable_existing_loggers=False if you're using Python 2.6 or later (see the docs for more information). The default value is True for backward compatibility, which causes all existing loggers to be disabled by fileConfig() unless they or their ancestor are explicitly named in the configuration. With the value set to False, existing loggers are left alone. If using Python 2.7/Python 3.2 or later, you may wish to consider the dictConfig() API which is better than fileConfig() as it gives more control over the configuration.

like image 60
Vinay Sajip Avatar answered Oct 10 '22 09:10

Vinay Sajip


Actually every logger is a child of the parent's package logger (i.e. package.subpackage.module inherits configuration from package.subpackage), so all you need to do is just to configure the root logger. This can be achieved by logging.config.fileConfig (your own config for loggers) or logging.basicConfig (sets the root logger). Setup logging in your entry module (__main__.py or whatever you want to run, for example main_script.py. __init__.py works as well)

using basicConfig:

# package/__main__.py import logging import sys  logging.basicConfig(stream=sys.stdout, level=logging.INFO) 

using fileConfig:

# package/__main__.py import logging import logging.config  logging.config.fileConfig('logging.conf') 

and then create every logger using:

# package/submodule.py # or # package/subpackage/submodule.py import logging log = logging.getLogger(__name__)  log.info("Hello logging!") 

For more information see Advanced Logging Tutorial.

like image 33
Stan Prokop Avatar answered Oct 10 '22 09:10

Stan Prokop