Collectstatic and S3: not finding updated files

Question

I am using Amazon S3 to store static files for a Django project, but collectstatic is not finding updated files - only new ones.

I've been looking for an answer for ages, and my guess is that I have something configured incorrectly. I followed this blog post to help get everything set up.

I also ran into this question which seems identical to my problem, but I have tried all the solutions already.

I even tried using this plugin which is suggested in this question.

Here is some information that might be useful:

settings.py

...
STATICFILES_FINDERS = (
    'django.contrib.staticfiles.finders.FileSystemFinder',
    'django.contrib.staticfiles.finders.AppDirectoriesFinder',
    'django.contrib.staticfiles.finders.DefaultStorageFinder',
)
...
# S3 Settings
AWS_STORAGE_BUCKET_NAME = os.environ['AWS_STORAGE_BUCKET_NAME']
STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
S3_URL = 'http://%s.s3.amazonaws.com/' % AWS_STORAGE_BUCKET_NAME
STATIC_URL = S3_URL
AWS_PRELOAD_METADATA = False

requirements.txt

...
Django==1.5.1
boto==2.10.0
django-storages==1.1.8
python-dateutil==2.1

Edit1:

I apologize if this question is too unique to my own circumstances to be of any help to a large audience. Nonetheless - this is has been hampering my productivity for a long time and I have wasted many hours looking for solutions, so I am starting a bounty to reward anyone who can help troubleshoot this problem.

Edit2:

I just ran across a similar problem somewhere. I am in a different timezone than the location of my AWS bucket. If by default collectstatic uses time stamp, could this interfere with the process?

Thanks

Answer 1

I think I solved this problem. Like you, I have spent so many hours on this problem. I am also on the bug report you found on bitbucket. Here is what I just accomplished.

I had

django-storages==1.1.8
Collectfast==0.1.11

This does not work at all. Delete everything for once does not work either. After that, it cannot pick up modifications and refuse to update anything.

The problem is with our time zone. S3 will say the files it has is last modified later than the ones we want to upload. django collectstatic will not try to copy the new ones over at all. And it will call the files "unmodified". For example, here is what I see before my fix:

Collected static files in 0:00:45.292022.
Skipped 407 already synced files.
0 static files copied, 1 unmodified.

My solution is "To the hell with modified time!". Besides the time zone problems that we are solving here, what if I made a mistake and need to roll back? It will refuse to deploy the old static files and leave my website broken.

Here is my pull request to Collectfast https://github.com/FundedByMe/collectfast/pull/11 . I still left a flag so if you really want to check modified time, you can still do it. Before it got merged, just use my code at https://github.com/sunshineo/collectfast

You have a nice day!

--Gordon PS: Stayed up till 4:40am for this. My day is ruined for sure.

Answer 2

After hours of digging around, I found this bug report.

I changed my requirements to revert to a previous version of Django storages.

django-storages==1.1.5