Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to use Python's RotatingFileHandler

Tags:

python

logging

I'm trying to do a test run of the logging module's RotatingFileHandler as follows:

import logging
from logging.handlers import RotatingFileHandler

# logging.basicConfig(filename="example.log", level=logging.DEBUG)

logger = logging.getLogger('my_logger')
handler = RotatingFileHandler("my_log.log", maxBytes=2000, backupCount=10)
logger.addHandler(handler)

for _ in range(10000):
    logger.debug("Hello, world!")

However, with logging.basicConfig line commented out, the resulting my_log.log file contains no data:

enter image description here

If I comment in the line with logging.basicConfig(filename="example.log", level=logging.DEBUG), I get the expected my_log.log files with numbered suffixes. However, there is also the example.log which is a (relatively) large file:

enter image description here

How can I set up the logging so that it only generates the my_log.log files, and not the large example.log file?

like image 926
Kurt Peek Avatar asked Oct 17 '16 14:10

Kurt Peek


People also ask

What is a RotatingFileHandler?

RotatingFileHandler allows a log file to grow up to size N, and then immediately and automatically rotates to a new file.

What is a log handler Python?

Python Logging Handler The log handler is the component that effectively writes/displays a log: Display it in the console (via StreamHandler), in a file (via FileHandler), or even by sending you an email via SMTPHandler, etc. Each log handler has 2 important fields: A formatter which adds context information to a log.


Video Answer


3 Answers

Python provides 5 logging levels out of the box (in increasing order of severity): DEBUG, INFO, WARNING, ERROR and CRITICAL. The default one is WARNING. The docs says, that

Logging messages which are less severe than lvl will be ignored.

So if you use .debug with the default settings, you won't see anything in your logs.

The easiest fix would be to use logger.warning function rather than logger.debug:

import logging
from logging.handlers import RotatingFileHandler

logger = logging.getLogger('my_logger')
handler = RotatingFileHandler('my_log.log', maxBytes=2000, backupCount=10)
logger.addHandler(handler)

for _ in range(10000):
    logger.warning('Hello, world!')

And if you want to change logger level you can use .setLevel method:

import logging
from logging.handlers import RotatingFileHandler

logger = logging.getLogger('my_logger')
logger.setLevel(logging.DEBUG)
handler = RotatingFileHandler('my_log.log', maxBytes=2000, backupCount=10)
logger.addHandler(handler)

for _ in range(10000):
    logger.debug('Hello, world!')
like image 189
skovorodkin Avatar answered Oct 19 '22 05:10

skovorodkin


Going off of Kurt Peek's answer you can also put the rotating file handler in the logging.basicConfig directly

import logging
from logging.handlers import RotatingFileHandler
logging.basicConfig(
        handlers=[RotatingFileHandler('./my_log.log', maxBytes=100000, backupCount=10)],
        level=logging.DEBUG,
        format="[%(asctime)s] %(levelname)s [%(name)s.%(funcName)s:%(lineno)d] %(message)s",
        datefmt='%Y-%m-%dT%H:%M:%S')
like image 52
Thomas Burke Avatar answered Oct 19 '22 07:10

Thomas Burke


All previous answers are correct, here another way of doing the same thing except we use logging config file instead.

logging_config.ini

Here is the config file :

[loggers]
keys=root

[handlers]
keys=logfile

[formatters]
keys=logfileformatter

[logger_root]
level=DEBUG
handlers=logfile

[formatter_logfileformatter]
format=%(asctime)s %(name)-12s: %(levelname)s %(message)s

[handler_logfile]
class=handlers.RotatingFileHandler
level=DEBUG
args=('testing.log','a',10,100)
formatter=logfileformatter

myScrypt.py

here is simple logging script that uses the above config file

import logging
from logging.config import fileConfig

fileConfig('logging_config.ini')
logger = logging.getLogger()
logger.debug('the best scripting language is python in the world')

RESULT

here is the result, notice maxBytes is set to 10 but in real life, that's clearly too small. (args=('testing.log','a',10,100)

enter image description here

like image 22
grepit Avatar answered Oct 19 '22 07:10

grepit