Speeding up deploys and development builds for image heavy Gatsby.js website

Question

I'm using Gatsby.js and gatsby-image to build a website that currently has about 300 images on it. I'm encountering 2 problems:

1. gatsby develop and gatsby build take many minutes to run because gatsby-image generates multiple resolutions and svg placeholder images for every image on the site. This makes for a great user experience once that pre-optimization is done, but a very slow development experience if I ever need to re-build.

My current workaround is to remove all but a few images during development.

2. Deploying to GitHub Pages takes too long with so many images (300 base images * 3 resolutions + 1 svg representation). Trying to deploy the website to GitHub pages causes a timeout. Going to attempt deploying to Netlify instead, but I anticipate the same problem. I also don't want to have to re-upload the images every time I make a change on the website.

I don't feel like my <1000 images should qualify as "image heavy", but given poor to mediocre upload speeds, I need a way to upload them incrementally and not re-upload images that have not changed.

Is there a way to upload images separately from the rest of a build for a Gatsby website?

I think maybe I could get something working with AWS S3, manually choosing which files from my build folder I upload when creating a new deploy.

Anyone else dealt with handling an image-heavy Gatsby site? Any recommendations for speeding up my build and deploy process?

Answer 1

Disclaimer: I work for Netlify

Our general recommendation is to do the image optimization locally and check those files into GitHub since it can take longer than our CI allows you (15 minutes) to do all that work and it is repetitive.

There is also an npm module that lets you cache the things you've made alongside your dependencies: https://www.npmjs.com/package/cache-me-outside that may do that for you without abusing GitHub (instead abusing Netlify's cache :))