Django log to file by date or hour

Question

I know django can log to one file, but I want know how to print to different files in diff times. File name like 20150101.log 20150102.log.

Two questions:

1. Can I implement this by LOGGING->filename config ?
2. if Q1 can't, How can I dynamic change handlers ?

---

In other words, How can I make django split file not by filesize but by datetime.

Answer 1

TimedRotatingFileHandler

To log to different files based on the date or time you can use the logging.handlers.TimedRotatingFileHandler class.

The TimedRotatingFileHandler logs all messages to a file with a name that you specify. After the designated interval has elapsed it will rotate the current file to a file with the given name with the date and time appended to it.

If your filename is myproject.log then at the end of each interval it will rotate that file to be myproject.log.2015-01-01 and then continue logging current messages to myproject.log.

Unfortunately, this class does not behave quite like you might expect. Rather than rotating based on the actual date and time, it rotates based on the time interval.

If your chosen interval is 1 day and you happen to start Django running at noon on January 1st it will not rotate at midnight and put all messages from January 1st into a file named myproject.log.2015-01-01. Rather, it will rotate at noon on January 2nd which is 1 day after you started. It will rotate all messages to a file named myproject.log.2015-01-02. So this file will contain messages from both January 1st and 2nd.

If you wish for the class to behave based on actual time it would be fairly trivial to override the class and specifically override the method called shouldRollover.

At least the class will give you daily intervals so you can quickly find the log files that you need. You will only have to look in two files.

To set it up in Django do the following:

LOGGING = { 
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'verbose': {
            'format' : "[%(asctime)s] %(levelname)s [%(name)s:%(lineno)s] %(message)s",
            'datefmt' : "%d/%b/%Y %H:%M:%S"
        },
        'simple': {
            'format': '%(levelname)s %(message)s'
        },
    },  
    'handlers': {
        'file': {
            'level': 'INFO',
            'class': 'logging.handlers.TimedRotatingFileHandler',
            'filename': 'myproject.log',
            'when': 'D', # this specifies the interval
            'interval': 1, # defaults to 1, only necessary for other values 
            'backupCount': 10, # how many backup file to keep, 10 days
            'formatter': 'verbose',
        },

    },  
    'loggers': {
        'django': {
            'handlers': ['file'],
            'level': os.getenv('DJANGO_LOG_LEVEL', 'INFO'),
        },
        '': {
            'handlers': ['file'],
            'level': os.getenv('DJANGO_LOG_LEVEL', 'INFO'),
        }
    },  
}

Answer 2

The TimedRotatingFileHandler class, located in the logging.handlers module, supports rotation of disk log files at certain timed intervals. Use the when to specify the type of interval. To view the list of possible values and get more information, visit source link.

Add the following to settings.py

LOGGING = {
    "version": 1,
    "disable_existing_loggers": False,
    "formatters": {
        "verbose": {
            "format": "{levelname} {asctime} [{name}:{lineno}] {message}",  # other options: {module} {process:d} {thread:d}
            "datefmt": "%Y-%m-%d_%H:%M:%S",
            "style": "{",
        },
        "simple": {
            "format": "{levelname} {message}",
            "style": "{",
        },
    },
    "handlers": {
        "file": {
            "level": "INFO",
            "class": "logging.handlers.TimedRotatingFileHandler",
            "formatter": "verbose",
            "filename": "myproject.log",
            "when": "midnight",  # specify the type of interval
            "interval": 1,
            "backupCount": 5,  # number of backup files to keep
            "encoding": None,
            "delay": False,
            "utc": False,
            "atTime": None,
            "errors": None,
        },
        "console": {
            "level": "INFO",
            "class": "logging.StreamHandler",
            "formatter": "simple",
        },
    },
    "loggers": {
        "django": {
            "handlers": ["console"],
            "propagate": True,
        },
        "": {
            "handlers": ["file"],
            "level": "INFO",
        },
    },
}

Define logger in the file you want to log as below:

import logging    
logger = logging.getLogger(__name__)

Then use this line of code to log:

logger.info("logging message")

Important notes:

• The system will save old log files by appending extensions to the filename. The extensions are date-and-time based (example: myproject.log --> myporject.log.2022-07-12)
• If the utc argument is true, times in UTC will be used; otherwise local time is used.
• If atTime is not None, it must be a datetime.time instance which specifies the time of day when rollover occurs, for the cases where rollover is set to happen “at midnight” or “on a particular weekday”.
• If errors is specified, it’s used to determine how encoding errors are handled.