Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Logging to multiple log files from different classes in Python

Tags:

python

logging

I want to write a Python class which uses Python logging. This Python class will be responsible for the creating a file with a given name in init function.

I want to create a object of the above class in two or more classes and expect two or files getting generated.

I tried writing this class but I am not able to create multiple files.

Can anyone guide me how do I do that?

I have created the following class:

class Logger: def __init__(self, log_filename = "test.log"):     if not os.path.exists("LogFiles"):         os.makedirs("LogFiles")     self.Logger = logging.getLogger("main")     logging.basicConfig(level=logging.DEBUG,                         format='%(asctime)s : %(message)s',                         filename= log_filename,                         filemode='w')           # change filemode to 'w' to overwrite file on each run      consoleHandler = logging.StreamHandler()     consoleHandler.setLevel(logging.DEBUG)     formatter = logging.Formatter('%(asctime)s - %(message)s')     consoleHandler.setFormatter(formatter)     logging.getLogger('').addHandler(consoleHandler)      # Add to the root logger     self.Logger.info("Starting new logging sessions")   def writeToFile(self, line):     if self.Logger.propagate == True:         self.Logger.debug(line)  def closeFile(self):      if self.Logger.propagate == True:         self.Logger.propagate = False 
like image 991
Shrikanth Kalluraya Avatar asked Jun 11 '13 01:06

Shrikanth Kalluraya


People also ask

How do I create a multiple logging level in Python?

You can set a different logging level for each logging handler but it seems you will have to set the logger's level to the "lowest". In the example below I set the logger to DEBUG, the stream handler to INFO and the TimedRotatingFileHandler to DEBUG. So the file has DEBUG entries and the stream outputs only INFO.

What are the five levels of logging in Python?

In Python, the built-in logging module can be used to log events. Log messages can have 5 levels - DEBUG, INGO, WARNING, ERROR and CRITICAL. They can also include traceback information for exceptions. Logs can be especially useful in case of errors to help identify their cause.


2 Answers

Sounds like the internals of your class should probably have a Logger and that you'll want to add a FileHandler to the Logger. You might want to consider just using a factory method that creates Loggers and adds the handler instead of implementing your own class. You may need to create the directories that hold the log files. See this answer for advice on creating directories.

Edit:

I don't think you need to write your own Logger class. Python's logging module has all the pieces you need. You probably just need a factory method. The key to realize is you need to create two separate, completely independent logging objects. You do this with logging.getLogger, and any time you pass it a different name, it gives you a different logger. You can use anything you want for the logger's name. For sure, you want to stay away from basicConfig for what you're doing. It's designed to be something simple for people who just want one Logger not doing anything too special.

I think this demonstrates the functionality you're after. The key is create two different loggers with different handlers. Then use them separately. Keep in mind that my second call to logging.getLogger doesn't create a new logger; it gets the one we set up initially in setup_logger.

log_test.py:

from __future__ import absolute_import  import logging  def setup_logger(logger_name, log_file, level=logging.INFO):     l = logging.getLogger(logger_name)     formatter = logging.Formatter('%(asctime)s : %(message)s')     fileHandler = logging.FileHandler(log_file, mode='w')     fileHandler.setFormatter(formatter)     streamHandler = logging.StreamHandler()     streamHandler.setFormatter(formatter)      l.setLevel(level)     l.addHandler(fileHandler)     l.addHandler(streamHandler)      def main():     setup_logger('log1', r'C:\temp\log1.log')     setup_logger('log2', r'C:\temp\log2.log')     log1 = logging.getLogger('log1')     log2 = logging.getLogger('log2')      log1.info('Info for log 1!')     log2.info('Info for log 2!')     log1.error('Oh, no! Something went wrong!')  if '__main__' == __name__:     main() 

Sample run:

C:\temp>C:\Python\27\python.exe logtest.py 2013-06-12 02:00:13,832 : Info for log 1! 2013-06-12 02:00:13,832 : Info for log 2! 2013-06-12 02:00:13,832 : Oh, no! Something went wrong! 

log1.log:

2013-06-12 02:00:13,832 : Info for log 1! 2013-06-12 02:00:13,832 : Oh, no! Something went wrong! 

log2.log:

2013-06-12 02:00:13,832 : Info for log 2! 
like image 142
jpmc26 Avatar answered Sep 16 '22 16:09

jpmc26


This is better handled using dictConfig. You can then specify logging to two seperate files. I utilize the second_logger only when an API post is made so that I can log external data to the second log.

import os, logging from logging.config import dictConfig  FORMAT = "%(asctime)s {app} [%(thread)d] %(levelname)-5s %(name)s - %(message)s. [file=%(filename)s:%(lineno)d]" DATE_FORMAT = None   def setup_logging(name, level="INFO", fmt=FORMAT):     formatted = fmt.format(app=name)     log_dir = r'C:/log_directory'     if not os.path.exists(log_dir):         os.makedirs(log_dir)      logging_config = {         "version": 1,         'disable_existing_loggers': False,         "formatters": {             'standard': {                 'format': formatted             }         },         "handlers": {             'default': {                 'class': 'logging.StreamHandler',                 'formatter': 'standard',                 'level': level,                 'stream': 'ext://sys.stdout'             },             'file': {                 'class': 'logging.handlers.TimedRotatingFileHandler',                 'when': 'midnight',                 'utc': True,                 'backupCount': 5,                 'level': level,                 'filename': '{}/app_manager.log'.format(log_dir),                 'formatter': 'standard',             },             'file2': {                 'class': 'logging.handlers.TimedRotatingFileHandler',                 'when': 'midnight',                 'utc': True,                 'backupCount': 5,                 'level': level,                 'filename': '{}/unified_log.log'.format(log_dir),                 'formatter': 'standard',             }         },         "loggers": {             "": {                 'handlers': ['default', 'file'],                 'level': level             },             "second_log": {                 'handlers': ['default', 'file2'],                 'level': level             }         }     }      dictConfig(logging_config)  log.setup_logging(name="log-name", level=LEVEL logger = logging.getLogger(__name__) second_logger = logging.getLogger('second_log') second_logger.propagate = False 
like image 32
Sherd Avatar answered Sep 16 '22 16:09

Sherd