Django collectstatic not working on production with S3, but same settings work locally

Question

I've been moving around some settings to make more defined local and production environments, and I must have messed something up.

Below are the majority of relevant settings. If I move the production.py settings (which just contains AWS-related settings at the moment) to base.py, I can update S3 from my local machine just fine. Similarly, if I keep those AWS settings in base.py and push to production, S3 updates appropriately. In addition, if I print something from production.py, it does print. However, if I make production.py my "local" settings on manage.py, or when I push to Heroku with the settings as seen below, S3 is not updating.

What about my settings is incorrect? (Well, I'm sure a few things, but specifically causing S3 not to update?)

Here's some relevant code:

__init__.py (in the directory with base, local, and production)

from cobev.settings.base import *

base.py

INSTALLED_APPS = [
    ...
    'whitenoise.runserver_nostatic',
    'django.contrib.staticfiles',
    ...
    'storages',
]

...

STATIC_URL = '/static/'

STATICFILES_DIRS = [os.path.join(BASE_DIR, "global_static"),
                    os.path.join(BASE_DIR, "media", )
                    ]

MEDIA_ROOT = os.path.join(BASE_DIR, 'media/')
MEDIA_URL = '/media/'

local.py

# local_settings.py
from .base import *

...

STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')

production.py

from .base import *

# AWS Settings

AWS_ACCESS_KEY_ID = config('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = config('AWS_SECRET_ACCESS_KEY')
AWS_STORAGE_BUCKET_NAME = 'cobev'

AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
AWS_S3_OBJECT_PARAMETERS = {
    'CacheControl': 'max-age=86400',
}
AWS_LOCATION = 'static'

AWS_DEFAULT_ACL = 'public-read'

STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
DEFAULT_FILE_STORAGE = 'cobev.storage_backends.MediaStorage'
STATIC_URL = 'https://%s/%s/' % (AWS_S3_CUSTOM_DOMAIN, AWS_LOCATION)

ADMIN_MEDIA_PREFIX = STATIC_URL + 'admin/'

# End AWS

wsgi.py

import os

from django.core.wsgi import get_wsgi_application

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "cobev.settings.production")

application = get_wsgi_application()

from whitenoise.django import DjangoWhiteNoise
application = DjangoWhiteNoise(application)

manage.py

#!/usr/bin/env python
import os
import sys

if __name__ == "__main__":
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "cobev.settings.local") 
    try:
        from django.core.management import execute_from_command_line
    except ImportError as exc:
        raise ImportError(
            "Couldn't import Django. Are you sure it's installed and "
            "available on your PYTHONPATH environment variable? Did you "
            "forget to activate a virtual environment?"
        ) from exc
    execute_from_command_line(sys.argv)

Answer 1

Ok, let me try, as discovered in comments from the question, you do S3 update using collectstatic, but this is a management command which is called using manage.py file where you set cobev.settings.local as settings which are not equal to cobev.settings.production which is used for wsgi.py file.

I think you should manage your settings file using, normal Django way, OS environment variable named DJANGO_SETTINGS_MODULE.

For sure you should be able to set it in any production environment you are running.

Answer 2

Then if that can help here is my config for AWS/Django/S3 in production :

Common Static config :

# STATIC FILE CONFIGURATION
# ------------------------------------------------------------------------------
# See: https://docs.djangoproject.com/en/dev/ref/settings/#static-root
STATIC_ROOT = str(ROOT_DIR('staticfiles'))

# See: https://docs.djangoproject.com/en/dev/ref/settings/#static-url
STATIC_URL = '/static/'

# See: https://docs.djangoproject.com/en/dev/ref/contrib/staticfiles/#std:setting-STATICFILES_DIRS
STATICFILES_DIRS = [
    str(APPS_DIR.path('static'))
]

# See: https://docs.djangoproject.com/en/dev/ref/contrib/staticfiles/#staticfiles-finders
STATICFILES_FINDERS = [
    'django.contrib.staticfiles.finders.FileSystemFinder',
    'django.contrib.staticfiles.finders.AppDirectoriesFinder'
]

Common Media config :

# MEDIA CONFIGURATION
# ------------------------------------------------------------------------------

# See: https://docs.djangoproject.com/en/dev/ref/settings/#media-root
MEDIA_ROOT = str(APPS_DIR('media'))

# See: https://docs.djangoproject.com/en/dev/ref/settings/#media-url
MEDIA_URL = '/media/'

Production Static config :

# STATIC CONFIG PRODUCTION
# ------------------------------------------------------------------------------
# See: http://django-storages.readthedocs.io/en/latest/backends/amazon-S3.html

AWS_STORAGE_BUCKET_NAME = 'mybucket-name-production'
AWS_ACCESS_KEY_ID = 'YOUR_KEY_ID'
AWS_SECRET_ACCESS_KEY = 'YOUR_SECRET_KEY'
AWS_S3_HOST = "s3.amazonaws.com"
AWS_S3_URL = 'https://{bucker_name}.s3.amazonaws.com/'.format(bucker_name=AWS_STORAGE_BUCKET_NAME)

AWS_LOCATION = 'static/'

AWS_S3_URL_PROTOCOL = 'https:'
AWS_S3_CUSTOM_DOMAIN = 'static.mydomain.com' # I use sub domaine to serve static 

STATIC_URL = '{AWS_S3_URL_PROTOCOL}//{AWS_S3_CUSTOM_DOMAIN}/{AWS_LOCATION}'.format(
    AWS_S3_URL_PROTOCOL=AWS_S3_URL_PROTOCOL,
    AWS_S3_CUSTOM_DOMAIN=AWS_S3_CUSTOM_DOMAIN,
    AWS_LOCATION=AWS_LOCATION)

AWS_QUERYSTRING_AUTH = False

AWS_IS_GZIPPED = True
AWS_EXPIREY = 60 * 60 * 24 * 14

# For s3boto
AWS_HEADERS = {
    'Cache-Control': 'max-age=%d, s-maxage=%d, must-revalidate' % (AWS_EXPIREY, AWS_EXPIREY)
}

# For s3boto3
AWS_S3_OBJECT_PARAMETERS = {
    'CacheControl': 'max-age=%d' % AWS_EXPIREY,
}

AWS_PRELOAD_METADATA = True
#AWS_S3_FILE_OVERWRITE = True

STATICFILES_STORAGE = 'config.storages.StaticStorage'
DEFAULT_FILE_STORAGE = 'config.storages.DefaultStorage'



# MEDIA S3 CONFIG PRODUCTION
# --------------------------------------------------------------------------------

AWS_MEDIA_DIR = 'media'
MEDIA_URL = AWS_S3_URL + AWS_MEDIA_DIR + '/'
MEDIA_ROOT = MEDIA_URL

Here is my StaticStorage Class :

from storages.backends.s3boto3 import S3Boto3Storage


class StaticStorage(S3Boto3Storage):
    location = 'static'
    file_overwrite = False


class DefaultStorage(S3Boto3Storage):
    location = ''
    file_overwrite = False

After That I add command in the folder .ebextensions in .config file for collectstatic :

# ./ebextensions/02_container_commands.config file : 

container_commands:
  0.3.0.push.static.to.s3:
    command: "source /opt/python/run/venv/bin/activate && python manage.py collectstatic --ignore=.scss --noinput"
    leader_only: true
    ignoreErrors: true