Serve Static files from Google Cloud Storage Bucket (for Django App hosted on GCE)

Question

I'm trying to serve the static Files for my django App from Cloud Storage Bucket but don't know the exact process. Can someone please suggest a proper way to do so ?

Steps I did:

1. Uploaded all the static files on Google Cloud Storage Bucket(www.example.com) using gsutil command.
2. Edited /etc/apache2/sites-available/default-ssl.conf File.

File Content:

<VirtualHost *:443>
        ServerName example.com
        ServerAdmin [email protected]

 #       Alias /static /opt/projects/example-google/example_static
        Alias /static https://storage.googleapis.com/www.example.com/static
        <Directory /opt/projects/example-google/example_static>
           Require all granted
        </Directory>

        <Directory /opt/projects/example-google/example/example>
            <Files wsgi.py>
                Require all granted
            </Files>
        </Directory>

        WSGIDaemonProcess example python-path=/opt/projects/example-google/example:/opt/projects/example-google/venv/lib/python2.7/site-packages
        WSGIProcessGroup example
WSGIApplicationGroup %{GLOBAL}
        WSGIScriptAlias / /opt/projects/example-google/example/example/wsgi.py

        SSLEngine on
        SSLCertificateFile  /etc/apache2/ssl/example.com.crt
        SSLCertificateKeyFile /etc/apache2/ssl/example.com.key
        SSLCertificateChainFile /etc/apache2/ssl/intermediate.crt
</VirtualHost>

settings.py File:

# Static files (CSS, JavaScript, Images)
STATIC_URL = '/static/'
# STATIC_URL = 'https://storage.googleapis.com/www.example.com/static/'
STATIC_ROOT = os.path.join(BASE_DIR, '../example_static')

MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, '../example_media')
STATICFILES_DIRS = (os.path.join(BASE_DIR, 'static'), MEDIA_ROOT,)

Any suggestion on what all additional changes are required for this task ?

Thanks,

Answer 1

Main references:

• https://django-storages.readthedocs.io/en/latest/backends/gcloud.html
• https://medium.com/@umeshsaruk/upload-to-google-cloud-storage-using-django-storages-72ddec2f0d05

Prerequisite steps

1. Go to GCP: Cloud Storage (GCS) and click on CREATE BUCKET (fill-up as needed)

2. Once created, you can make it public if you want it to act like a CDN of your website (storage of your static files such as css, images, videos, etc.)

   • Go to your newly created bucket
   • Go to Permissions and then Click Add members
   • Add a new member "allUsers" with role "Cloud Storage - Storage Object Viewer"
   • Reference: https://cloud.google.com/storage/docs/quickstart-console

Method 1 (easier and faster, but requires constant manual copying of files to the GCS)

1. Configure your Django's static file settings in your settings.py

# Tell Django about the different locations to where the static files used by the project can be found
STATICFILES_DIRS = [
    os.path.join(BASE_DIR, 'templates'),
    os.path.join(BASE_DIR, "yourapp1", "templates"),
    os.path.join(BASE_DIR, "yourapp2", "static"),
    os.path.join(BASE_DIR, "watever"),
    "/home/me/Music/TaylorSwift/",
    "/home/me/Videos/notNsfw/",
]

# If the command "collectstatic" is invoked, tell Django where to place all the collected static
# files from all the directories included in STATICFILES_DIRS. Be aware that configuring it with a
# path outside your /home/me means that you need to have permissions to write to that folder later
# on when you invoke "collectstatic", so you might need to login as root first or run it as sudo.
STATIC_ROOT = "/var/www/mywebsite/"

# Tell Django the base url to access the static files. Think of this as the "prefix" of the URL
# to where your static files are. Note that if you browse through your bucket and happen to see a
# URL such as "https://storage.cloud.google.com/<your_bucket_name>/someFileYouHaveUploaded", such
# URL requires that whoever accesses it should be currently logged-in with their Google accounts. If
# you want your static files to be publicly accessible by anyone whether they are logged-in or not,
# use the link "https://storage.googleapis.com/<your_bucket_name>/someFileYouHaveUploaded" instead.
STATIC_URL = "https://storage.googleapis.com/<your_bucket_name>/"

# References:
# https://docs.djangoproject.com/en/3.0/howto/static-files/
# https://docs.djangoproject.com/en/3.0/howto/static-files/deployment/
# https://docs.djangoproject.com/en/3.0/ref/settings/

2. If you have HTML files or CSS files that access other static files, make sure that they reference those other static files with this updated STATIC_URL setting.

In your home.html

{% load static %}
<link rel="stylesheet" type="text/css" href="{% static 'home/css/home.css' %}">

Then in your home.css file

background-image: url("../assets/img/myHandsomeImage.jpg");

The home.css link now would translate to:

https://storage.googleapis.com/[your_bucket_name]/home/css/home.css

While the myHandsomeImage.jpg would be:

https://storage.googleapis.com/[your_bucket_name]/home/assets/img/myHandsomeImage.jpg

Of course if you wish, you could just put the absolute path (complete URL), but such configuration would always require you to update the used URLs manually, like if you switched to development mode and wanted to just access the static files locally instead of from GCS.

3. Run below. This would copy all files from each directory in STATICFILES_DIRS to STATIC_ROOT directory.

python3 manage.py collectstatic

# or if your STATIC_ROOT folder requires permissions to write to it then:
# sudo python3 manage.py collectstatic

4. Go to the STATIC_ROOT folder and upload its contents to GCS. Either upload them manually through the GCS GUI Console or through the Google provided tool "gsutil" along with rsync

5. Now, your GCS bucket already contains your static files, with your Django project configured to directly access those files through the configured STATIC_URL.

Method 2 (longer, but do not require manual copying after)

1. [OPTIONAL STEP] Prepare your python virtual environment

python3 -m venv path/to/the/target/location/for/the/virtual/environment
source path/to/the/target/location/for/the/virtual/environment/bin/activate

2. Install the necessary packages to be able to access and store directly to your GCS

pip3 install django-storages # https://pypi.org/project/django-storages/
pip3 install google-cloud-storage # https://pypi.org/project/google-cloud-storage/

3. [MANDATORY STEP if you are on a computer outside the Google Infrastructure] Go to GCP: IAM, Service Accounts, and click on CREATE SERVICE ACCOUNT

   • Name: SomeName
   • ID / email: somename
   • Role: Project - Owner
   • CREATE KEY, select JSON
   • Store the downloaded JSON file. This generated json key would be used later for authentication purposes once we start accessing and storing to the GCS
   • Reference: https://cloud.google.com/docs/authentication/getting-started

4. Configure your Django's static file settings in your settings.py

STATICFILES_DIRS = ['same_values_as_in_method_1_above']
DEFAULT_FILE_STORAGE = 'storages.backends.gcloud.GoogleCloudStorage'
GS_BUCKET_NAME = 'your_bucket_name'
STATICFILES_STORAGE = 'storages.backends.gcloud.GoogleCloudStorage'
STATIC_URL = 'https://storage.googleapis.com/<your_bucket_name>/'
from google.oauth2 import service_account
GS_CREDENTIALS = service_account.Credentials.from_service_account_file(
    'path/to/the/downloaded/json/key/credentials.json' # see step 3
)

# There are 2 ways to authenticate, you could either do 1 of the following
# 1. Define the variable GS_CREDENTIALS in the settings.py (as done above), or just
# 2. write the command "export GOOGLE_APPLICATION_CREDENTIALS='path/to/credentials.json'" in the shell where you would run the "collectstatic" command

5. Run below. This would copy all files from each directory in STATICFILES_DIRS directly to your GCS bucket. This might take a while.

python3 manage.py collectstatic

6. Now, your GCS bucket already contains your static files, with your Django project configured to directly access those files through the configured STATIC_URL.

Answer 2

Basically you need to:

1. create cloud storage bucket and set it to public readable.
2. collect static file local
3. copy file to cloud storage
4. set STATIC_URL

Check the step 1-4 https://cloud.google.com/python/django/container-engine#deploy_the_app_to_container_engine