Copy files from remote server to google cloud bucket

Question

How would I copy files from a remote server to a google bucket? For example,

gcloud compute scp username@server:/path/to/file gs://my-bucket

This method gives the error: All sources must be local files when destination is remote.

Additionally, gsutil only provides support for cp and not scp.

Thanks in advance!

Answer 1

You can also directly execute gsutil command on your GCE VM (Most VM images have Cloud SDK preinstalled). For example:

gcloud compute ssh user@server --zone my_zone \
  --command='gsutil cp path/to/my_file gs://MY_BUCKET'

Note that for this to work your service account associated with VM must have appropriate access scope to GCS. If you run

gcloud beta compute instances describe my_instance --zone my_zone \
  --format="value(serviceAccounts.scopes)"

It will show list of scopes set for VM service account. Make sure you have https://www.googleapis.com/auth/cloud-platform or https://www.googleapis.com/auth/devstorage.full_control or https://www.googleapis.com/auth/devstorage.read_write. If not you can use set-scopes beta command to reset them or go to console and edit VM in question.