Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Django logging during migration

I have numerous complex data migrations set up in Django and would like to set up logging to see how they complete and catch errors. I could do this with regular python logging but Django has this semi-built-in and I was hoping the Django logging would work with migrations. However the following setup does not seem to work. Where am I going wrong or does Django logging only work during serve and test?

In a migration file:

import logging
logger = logging.getLogger('rdm')
stuff
logger.warning('message')
more stuff

In Settings.py:

LOGGING = {
     'version': 1,
     'disable_existing_loggers': False,
     'filters': {
         'require_debug_false': {
             '()': 'django.utils.log.RequireDebugFalse',
         },
         'require_debug_true': {
             '()': 'django.utils.log.RequireDebugTrue',
         },
     },
     'formatters': {
         'simple': {
             'format': '[%(asctime)s] %(levelname)s %(message)s',
             'datefmt': '%Y-%m-%d %H:%M:%S'
         },
     },
     'handlers': {
         'rdm_logfile': {
             'level': 'DEBUG', # should capture everything (warning, info, etc.)
             'filters': ['require_debug_false','require_debug_true'],
             'class': 'logging.handlers.RotatingFileHandler',
             'filename': os.path.join(BASE_DIR,'django_rdm.log'),
             'maxBytes': 1024*1024*100, # 100MB
             'backupCount': 5,
             'formatter': 'simple'
         },
    },
    'loggers': {
         'rdm': {
         'handlers': ['rdm_logfile'],
         },
    }
}
like image 330
Austin Fox Avatar asked Jul 19 '18 00:07

Austin Fox


People also ask

How do I enable logging in Django?

By default, the LOGGING setting is merged with Django's default logging configuration using the following scheme. If the disable_existing_loggers key in the LOGGING dictConfig is set to True (which is the dictConfig default if the key is missing) then all loggers from the default configuration will be disabled.

How does Django keep track of migrations?

Django keeps track of applied migrations in the Django migrations table. Django migrations consist of plain Python files containing a Migration class. Django knows which changes to perform from the operations list in the Migration classes. Django compares your models to a project state it builds from the migrations.

What is the difference between Makemigrations and migrate in Django?

makemigrations is responsible for packaging up your model changes into individual migration files - analogous to commits - and migrate is responsible for applying those to your database.

What are migrations in Django?

Migrations are Django’s way of propagating changes you make to your models (adding a field, deleting a model, etc.) into your database schema. They’re designed to be mostly automatic, but you’ll need to know when to make migrations, when to run them, and the common problems you might run into.

How does system logging work in Python with Django?

Django uses and extends Python’s builtin logging module to perform system logging. This module is discussed in detail in Python’s own documentation; this section provides a quick overview. A logger is the entry point into the logging system. Each logger is a named bucket to which messages can be written for processing.

Is it possible to enable request logging in Django?

One nicety in Django's default config is request logging with runserver. By overridding Django's config, we lose it, but it is easy enough to add back in:

How to test forward and rollback migrations in Django?

You can test forward and rollback migrations and their ordering with the help of django-test-migrations. It is simple, friendly, and already works with the test framework of your choice. I also want to say “thank you” to these awesome people. Without their work it would take me much longer to come up with the working solution.


Video Answer


2 Answers

I had the same problem. Somehow I had to re-configure the base logger 'Django':

from django.conf import settings
import logging
logger = logging.getLogger(__name__)


def your_migration(apps, schema_editor):
    logger.setLevel(logging.INFO)
    settings.LOGGING['loggers']['django'] = {
        'level': 'INFO',
        'handlers': ['console']
    }
    logger.info("Test")
like image 175
Barbara Krausz Avatar answered Nov 15 '22 01:11

Barbara Krausz


I found this blog post to be helpful.

The tl;dr is that you can configure a root logger to catch everything that isn't otherwise specified. This (apparently) applies to all code run via manage.py including migrations, etc.

Root loggers have an empty name, i.e.:

    'loggers': {
         '': {
         'handlers': ['rdm_logfile'],
         },
like image 23
Owen Avatar answered Nov 15 '22 00:11

Owen