Serving gzipped content from django

Question

I'm trying to serve a gzipped version of a text/html page in Django, but Firefox is telling me there's a content encoding error.

NOTES:

• I realize this is not a best practice and I'm most likely going to use mod_gzip. This is just a learning exercise to understand what's going on.
• I know about the Django gzip middleware-- it has problems with binary files.

Here's my code:

rendered_page =  zlib.compress(template.render(context).encode('utf-8'))

response = HttpResponse(rendered_page)
response['Content-Encoding'] = 'gzip'
response['Content-Length'] = len(rendered_page)
return response

Am I missing something here? Is it possible that the content length is wrong? Are there additional headers I'm missing?

Thanks.

Answer 1

You could also simply use Django's GZip Middleware:

Either by enabling the middleware in settings.py by adding:

MIDDLEWARE_CLASSES = (
    django.middleware.gzip.GZipMiddleware,
    ...
)

Or do it before you return a particular response. In your views.py, dec would be the handler for a certain url

from django.middleware.gzip import GZipMiddleware

gzip_middleware = GZipMiddleware()

 def dec(request, *args, **kwargs):
        response = func(request, *args, **kwargs)
        return gzip_middleware.process_response(request, response)
        return dec

NOTE: You should be certain you are not subject to side-channel attacks before using GZip middleware.

Warning

Security researchers recently revealed that when compression techniques (including GZipMiddleware) are used on a website, the site may become exposed to a number of possible attacks. Before using GZipMiddleware on your site, you should consider very carefully whether you are subject to these attacks. If you’re in any doubt about whether you’re affected, you should avoid using GZipMiddleware. For more details, see the the BREACH paper (PDF) and breachattack.com.

Also:

Changed in Django 1.10: In older versions, Django’s CSRF protection mechanism was vulnerable to BREACH attacks when compression was used. This is no longer the case, but you should still take care not to compromise your own secrets this way.

Answer 2

If you're gzipping single page, not for all pages, you can use gzip_page decorator instead of GzipMiddleware.

from django.views.decorators.gzip import gzip_page

@gzip_page
def viewFunc(request):
  return HttpResponse("hello"*100)

Reference here: https://docs.djangoproject.com/en/1.4/topics/http/decorators/#module-django.views.decorators.gzip

Answer 3

zlib is a bit too low-level for this purpose. Here's how the GZip middleware itself does it (see compress_string in django.utils.text.py):

import cStringIO, gzip
zbuf = cStringIO.StringIO()
zfile = gzip.GzipFile(mode='wb', compresslevel=6, fileobj=zbuf)
zfile.write(template.render(context).encode('utf-8'))
zfile.close()

compressed_content = zbuf.getvalue()
response = HttpResponse(compressed_content)
response['Content-Encoding'] = 'gzip'
response['Content-Length'] = str(len(compressed_content))
return response

GZip uses zlib, but on its own zlib produces content that's improperly encoded for a browser seeing 'gzip' as the content encoding. Hope that helps!

Answer 4

For the sake of others finding this question and who are using nginx, this SO worked for me:

https://stackoverflow.com/a/41820704/4533488

Basically turning gzip on in the /etc/nginx/nginx.conf file did all the compression handling for me. On the client-side, most modern browsers automatically handle extracting (uncompressing) the data when receiving it - sweet!

Here is the nginx.conf file settings:

    http {

        #... other settings ...#

        ##
        # Gzip Settings
        ##

        gzip on;
        gzip_disable "msie6";

        gzip_vary on;
        gzip_proxied any;
        gzip_comp_level 6;
        gzip_buffers 16 8k;
        gzip_http_version 1.1;
        gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
    }