Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How copy file from buckte GCS to my local machine

I need copy files from Google Cloud Storage to my local machine:

I try this command o terminal of compute engine:

$sudo gsutil cp -r gs://mirror-bf /var/www/html/mydir

That is my directory on local machine /var/www/html/mydir.

i have that error:

CommandException: Destination URL must name a directory, bucket, or bucket subdirectory for the multiple source form of the cp command.

Where the mistake?

like image 766
MagicHat Avatar asked Feb 13 '18 22:02

MagicHat


2 Answers

You must first create the directory /var/www/html/mydir.

Then, you must run the gsutil command on your local machine and not in the Google Cloud Shell. The Cloud Shell runs on a remote machine and can't deal directly with your local directories.

like image 101
Brandon Yarbrough Avatar answered Oct 14 '22 10:10

Brandon Yarbrough


I have had a similar problem and went through the painful process of having to figuring it out too, so I thought I would provide my step by step solution (under Windows, hopefully similar for unix users) with snapshots and hope it helps others:

The first thing (as many others have pointed out on various stackoverflow threads), you have to run a local Console (in admin mode) for this to work (ie. do not use the cloud shell terminal).

Here are the steps:

  1. Assuming you already have Python installed on your machine, you will then need to install the gsutil python package using pip from your console:

    pip install gsutil

The Console looks like this:
enter image description here

  1. You will then be able to run the gsutil config from that same console:

    gsutil config

As you can see from the snapshot bellow, a .boto file needs to be created. It is needed to make sure you have permissions to access your drive.

enter image description here

Also note that you are now provided an URL, which is needed in order to get the authorization code (prompted in the console).

  1. Open a browser and paste this URL in, then:
  • Log in to your Google account (ie. account linked to your Google Cloud)

  • Google ask you to confirm you want to give access to GSUTIL. Click Allow:
    enter image description here

  • You will then be given an authorization code, which you can copy and paste to your console:
    enter image description here

  • Finally you are asked for a project-id:
    enter image description here

  1. Get the project ID of interest from your Google Cloud.
    In order to find these IDs, click on "My First Project" as circled here below:
    enter image description here

Then you will be provided a list of all your projects and their ID.
enter image description here

Paste that ID in you console, hit enter and here you are! You now have created your .boto file. This should be all you need to be able to play with your Cloud storage.
Console output:

Boto config file "C:\Users\xxxx\.boto" created. If you need to use a proxy to access the Internet please see the instructions in that file.
  1. You will then be able to copy your files and folders from the cloud to your PC using the following gsutil Command:

    gsutil -m cp -r gs://myCloudFolderOfInterest/ "D:\MyDestinationFolder"

Files from within "myCloudFolderOfInterest" should then get copied to the destination "MyDestinationFolder" (on your local computer).

like image 27
toughQuestions Avatar answered Oct 14 '22 11:10

toughQuestions