How do I use Django, sorl-thumbnail, memcached, and S3 together?

Question

I've got a project in which I need to start creating dynamically-resized thumbnails of user-uploaded images, where previously I'd been generating some specifically sized ones. I have an existing model, which represents an image which looks like this:

class Image(models.Model):
  original = models.URLField()
  small = models.URLField()
  medium = models.URLField()
  large = models.URLField()

Each of those is filled in with a URL to the respective image on S3. At the time of upload, all the versions are created (actually shortly after, via a scheduled task), stored in S3, and then set on the model. Going forward, our design needs many different sizes for each thumbnail, so storing them all in the db is not practical. I'd like to just keep original and do the rest on-the-fly in the templates.

So this is where I come to sorl-thumbnail. It seems to be intended to do what I want, but I'm kind of lost as to how it's supposed to be working (it's not at the moment). As a test I've taken made a basic view/template which lists some images and attempts to thumbnail them, much like the tutorial:

{% for img in image_set %}
  {% thumbnail img.original "180x180" crop="center" format="PNG" as im %}
  <img src="{{ im.url }}" width="{{ im.width }}" height="{{ im.height }}">
  {% endthumbnail %}
{% endfor %}

That ends up generating a URL for the file like cache/84/8f/848fb078f2e8d35aecd92150c6fa6843.png (which 404s). I'm not sure I understand where this file is supposed to be getting stored. I am assuming it is pulling in the S3 file and then creating the thumbnail somewhere, but I am unclear where that is, and where I am supposed to be serving it from. This is all on the local devserver right now, using memcached (in which I can see is getting cache entries set). PIL is installed in my environment with jpeg/png support.

Answer 1

You might want to look at Django-storages it was a nice AWS S3 that would fix the issue for you.Along with that also install boto, as django-storage has a dependency on boto.

Then you will have to add the following into your settings.py

import os

AWS_ACCESS_KEY_ID = os.environ.get('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = os.environ.get('AWS_SECRET_ACCESS_KEY')
AWS_STORAGE_BUCKET_NAME = '<YOUR BUCKET NAME>'

STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage'

STATIC_URL = 'http://' + AWS_STORAGE_BUCKET_NAME + '.s3.amazonaws.com/'
ADMIN_MEDIA_PREFIX = STATIC_URL + 'admin/'

Then your cache will be stored on AWS S3 itself.

I hope it helps

Note: for security reasons its a good idea to add your AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY as environment variables instead of just writing them down in setting.py directly.