django-storages and amazon s3 - suspiciousoperation

Question

I'm using django-storages with Amazon S3. I see the following error somewhat intermittently:

name = self._normalize_name(self._clean_name(name))\n\n  File \"/app/.heroku/venv/lib/python2.7/site-packages/storages/backends/s3boto.py\", line 237, in _normalize_name\n    name)\n\nSuspiciousOperation: Attempted access to 'https:/plantvillage.s3.amazonaws.com/avatar/hans9_avatar.jpg'

Note the single / after https:.

Does anyone know why this shows up? It doesn't happen all the time. I can successfully do this in other cases.

Answer 1

_normalize_name does a lot of fancy and mostly unnecessary on Django stuff with the URL. In my case I just override the S3BotoStorage like this:

class S3CustomStorage(S3BotoStorage):
def _normalize_name(self, name):
    """
    Get rid of this crap: http://stackoverflow.com/questions/12535123/django-storages-and-amazon-s3-suspiciousoperation
    """
    return name

Then use it in the storage property:

ImageField(storage=S3CustomStorage())

And it worked for django simple ImageField with this base configuration:

AWS_ACCESS_KEY_ID = 'TTTT'
AWS_SECRET_ACCESS_KEY = 'XXXX'
AWS_STORAGE_BUCKET_NAME = 'ZZZZ'

Answer 2

When you use default_storage methods make sure to use the file.name:

Correct:

default_storage.delete(file.name)

Wrong:

default_storage.delete(file.url)

Wrong:

default_storage.delete(file)

All three examples above work with local files, but when using s3 you will run into this error unless you use file.name.