gsutil rsync with gzip compression

Question

I'm hosting publicly available static resources in a google storage bucket, and I want to use the gsutil rsync command to sync our local version to the bucket, saving bandwidth and time. Part of our build process is to pre-gzip these resources, but gsutil rsync has no way to set the Content-Encoding header. This means we must run gsutil rsync, then immediately run gsutil setmeta to set headers on all the of gzipped file types. This leaves the bucket in a BAD state until that header is set. Another option is to use gsutil cp, passing the -z option, but this requires us to re-upload the entire directory structure every time, and this includes a LOT of image files and other non-gzipped resources that wastes time and bandwidth.

Is there an atomic way to accomplish the rsync and set proper Content-Encoding headers?

Answer 1

Assuming you're starting with gzipped source files in source-dir you can do:

gsutil -h content-encoding:gzip rsync -r source-dir gs://your-bucket

Note: If you do this and then run rsync in the reverse direction it will decompress and copy all the objects back down:

gsutil rsync -r gs://your-bucket source-dir 

which may not be what you want to happen. Basically, the safest way to use rsync is to simply synchronize objects as-is between source and destination, and not try to set content encodings on the objects.