Collectstatic creates empty files

Question

I'm trying to upgrade an app to Django 1.11, but experience issues with collectstatic.

Old versions:

django 1.8.17 
django-storages 1.5.1

New versions:

django 1.11.12
django-storages 1.6.6

Storage:

class StaticS3BotoStorage(ManifestFilesMixin, S3BotoStorage):
    location = 'static'
    file_overwrite = True
    preload_metadata = True

or

class StaticS3BotoStorage(CachedFilesMixin, S3BotoStorage):
    location = 'static'
    file_overwrite = True
    preload_metadata = True

With the old versions, collectstatic worked fine, including collectstatic --clear.

After the upgrade, collectstatic --clear fails (no files are deleted). collectstatic does copy files, however, sometimes it creates two versions of the same file. In this particular example, I get base.hash1.css and base.hash2.css. base.hash2.css is empty, so the pages open, but do not render correctly.

If I don't use CachedFilesMixin or ManifestFilesMixin, collectstatic works fine, but clear still fails.

I tested different combinations of django 1.11 and django-storages, but they all seem to behave the same.

Did someone else experience a similar issue?

Answer 1

We've encountered the same issue.

The underlying problem, I think, has multiple issues / sources:

• The ManifestFilesMixin uses and reuses ContentFile objects for generating the hashed files and saves them multiple times. Without resetting the ContentFile objects (by calling .seek(0) on them).
• The S3BotoStorage saves these files, without checking if they are at the correct position. Compare this to FileSystemStorage: The files are always read through from the beginning by iterating over the .chuncks() of the file.

We worked around the empty-file-issue by overriding S3BotoStorage like this:

class PatchedS3StaticStorage(S3BotoStorage):
    def _save(self, name, content):
        if hasattr(content, 'seek') and hasattr(content, 'seekable') and content.seekable():
            content.seek(0)
        return super()._save(name, content)

In short, we seek to the beginning of the file before saving it.