How to do gsutil cp -R while ignoring files like .git, .gitignore?

Question

I am trying to automate the process of syncing my web assets with Google Cloud Storage. I basically need to copy everything in my development directory up to the cloud. However, I need to ignore the .git directory and some other irrelevant files.

I can't just do a gsutil cp -R . <dest> because that takes absolutely everything, including .git. I tried find . | fgrep git | gsutil cp -I <dest> but that flattens all directories and puts them in root!

Is there a way I can solve this with gsutil or do I have to do a loop in script which uploads all directories (except .git) with -R and then uploads individual files in current directory?

Answer 1

You could use a command like:

gsutil rsync -x '\.git.*' dev_dir gs://your-bucket

See Google Storage - rsync - Synchronize content of two buckets/directories

Answer 2

You have two options:

A) Remove the git files after they are uploaded:

gsutil rm gs://bucket/\*.git\*

B) Use find to exclude git files:

find . -not -path '*/.git' -type f -printf '%P\n' | xargs -I '{}' gsutil cp '{}' gs://bucket/'{}'

Source: https://groups.google.com/forum/#!topic/gsutil-discuss/zoHhkTPhiNc

It would've been much easier if gsutil implemented rsync, this would've been easier with their --exclude flag.