django-storage with s3-boto break browser cache

Question

I have a django project which use django-storage over s3-boto.

Problem is that every file that is located on S3 unable to cache because the url is changed from each call.

here are two calls generated by django-storage :

https://my.s3.amazonaws.com/cache/user_6/profile_pic/profile_profile_picture_thumbnail.jpg?Signature=HlVSayUIJj6dMyk%2F4KBtFlz0uJs%3D&Expires=1364418058&AWSAccessKeyId=[awsaccesskey]     
https://my.s3.amazonaws.com/cache/user_6/profile_pic/profile_profile_picture_thumbnail.jpg?Signature=xh2VxKys0pkq7yHpbJmH000wkwg%3D&Expires=1364418110&AWSAccessKeyId=[awsaccesskey]

As you can see the signature is different. What can I do so it wont break my browser cache ?

Answer 1

In your settings, just add the following:

AWS_QUERYSTRING_AUTH = False

This will make sure that the URLs to the files are generated WITHOUT the extra parameters. Your URLs would look like:

https://my.s3.amazonaws.com/cache/user_6/profile_pic/profile_profile_picture_thumbnail.jpg

Answer 2

When AWS_QUERYSTRING_AUTH = True (which is the default), django will generate a temporary url each time we fetch the url.

If you don't want to generate a temporary url:

Add AWS_QUERYSTRING_AUTH = False to your settings.py

If you still want a temporary url:

Temporary urls will be valid for AWS_QUERYSTRING_EXPIRE seconds (3600 by default). So we can cache this temporary url (as long as we don't cache it longer than it's valid). This way - we can return the same url for subsequent page requests, allowing the client browser to fetch from their cache.

settings.py

# We subclass the default storage engine to add some caching
DEFAULT_FILE_STORAGE = 'project.storage.CachedS3Boto3Storage'

project/storage.py

import hashlib

from django.conf import settings
from django.core.cache import cache
from storages.backends.s3boto3 import S3Boto3Storage

class CachedS3Boto3Storage(S3Boto3Storage):
    """ adds caching for temporary urls """

    def url(self, name):
        # Add a prefix to avoid conflicts with any other apps
        key = hashlib.md5(f"CachedS3Boto3Storage_{name}".encode()).hexdigest()
        result = cache.get(key)
        if result:
            return result

        # No cached value exists, follow the usual logic
        result = super(CachedS3Boto3Storage, self).url(name)

        # Cache the result for 3/4 of the temp_url's lifetime.
        try:
            timeout = settings.AWS_QUERYSTRING_EXPIRE
        except:
            timeout = 3600
        timeout = int(timeout*.75)
        cache.set(key, result, timeout)

        return result