django - how to filter out GET static and media messages with logging?

Question

I'm working on a page that has lots of images so this generates lots of output in the console of this type. In my dev environment I use django to serve static and media, so I get a LOT of this in my console:

...
[23/May/2014 12:41:54] "GET /static/css/style.css HTTP/1.1" 304 0
[23/May/2014 12:41:55] "GET /static/js/jquery-1.7.1.min.js HTTP/1.1" 304 0
[23/May/2014 12:41:55] "GET /static/js/jquery.form.js HTTP/1.1" 304 0
...
[23/May/2014 12:41:57] "GET /media/producto/Tapa_Santiago_Vazquez_SV.jpg HTTP/1.1" 304 0
[23/May/2014 12:41:57] "GET /media/CACHE/images/producto/Barcos_y_mariposas_DVD_baja/2e3e3894ca08f88c03459e00f9018427.jpg HTTP/1.1" 304 0
[23/May/2014 12:41:56] "GET /media/CACHE/images/producto/tapaDEJA_VU/fb67e92ffd47808a263db02ca016bc24.jpg HTTP/1.1" 304 0
...

making it very tedious to look for meaningful output.

I would like to filter out those messages in my environment so I only see the GET for the view and my output, but so far looking at the logging I saw that I could affect other logging from django but not this. I even tried this but it didn't work:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': True,
    'handlers': {
        'null': {
            'level': 'ERROR',
            'class': 'django.utils.log.NullHandler',
        },
    },
    'loggers': {
        'django': {
            'handlers': ['null'],
            'level': 'ERROR',
            'propagate': True,
        },
    }
}

is it even possible to filter that kind of output out?

Thanks!!

Answer 1

Recent versions of Django make it really easy to override the default logging with your own LOGGING settings.

To filter out all GET requests to the /static/ directory, add the following to your settings.py:

def skip_static_requests(record):
    return not record.args[0].startswith('GET /static/')

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'filters': {
        # use Django's built in CallbackFilter to point to your filter 
        'skip_static_requests': {
            '()': 'django.utils.log.CallbackFilter',
            'callback': skip_static_requests
        }
    },
    'formatters': {
        # django's default formatter
        'django.server': {
            '()': 'django.utils.log.ServerFormatter',
            'format': '[%(server_time)s] %(message)s',
        }
    },
    'handlers': {
        # django's default handler...
        'django.server': {
            'level': 'INFO',
            'filters': ['skip_static_requests'],  # <- ...with one change
            'class': 'logging.StreamHandler',
            'formatter': 'django.server',
        },
    },
    'loggers': {
        # django's default logger
        'django.server': {
            'handlers': ['django.server'],
            'level': 'INFO',
            'propagate': False,
        },
    }
}

Here are all of Django's default loggers as of 1.10, which you can override in the same way: https://github.com/django/django/blob/32265361279b3316f5bce8efa71f2049409461e3/django/utils/log.py#L18

Here are descriptions of what Django's default built-in loggers do: https://docs.djangoproject.com/en/1.10/topics/logging/#id3

Here's are the docs on Django's CallbackFilter used above to hook in the custom filter: https://docs.djangoproject.com/en/1.10/topics/logging/#django.utils.log.CallbackFilter

Answer 2

As a workaround, you can use this snippet (from Django Snippets):

from django.conf import settings
from django.core.servers import basehttp
from django.core.management.commands.runserver import Command as BaseCommand

class QuietWSGIRequestHandler(basehttp.WSGIRequestHandler):
    def log_message(self, format, *args):
        # Don't bother logging requests for paths under MEDIA_URL.
        if self.path.startswith(settings.MEDIA_URL):
            return
        # can't use super as base is old-style class, so call method explicitly
        return basehttp.WSGIRequestHandler.log_message(self, format, *args)

def run(addr, port, wsgi_handler):
    server_address = (addr, port)
    httpd = basehttp.WSGIServer(server_address, QuietWSGIRequestHandler)
    httpd.set_app(wsgi_handler)
    httpd.serve_forever()

class Command(BaseCommand):
    def handle(self, addrport='', *args, **options):
        # monkeypatch Django to use our quiet server
        basehttp.run = run
        return super(Command, self).handle(addrport, *args, **options)

You need to user this command to run the server. It basically only override the log behaviour to drop requests which begins by the MEDIA_URL setting. Put this whole file in an installed app and run it with ./manage.py runserver_quiet (if the command file is runserver_quiet.py)

You can't play with the logging module, because the WSGIRequestHandler doesn't use it to display these messages. It is directly written in the stderr stream.

Answer 3

From Django 1.10 you can configure django.server logging for "the handling of requests received by the server invoked by the runserver command". This is a quick way to filter out the media requests and to focus on your logger info.

Add to django logging settings:

loggers: {
....
    'django.server': {
        'handlers': ['console'],
        'level': 'ERROR'  # or INFO if you want to see the log
    },
}

Django Documentation ref: Logging: django-server