How do I enable Google Cloud Storage on my local development server?

Question

I want to use a GCS bucket as the backing for my blobstore but I cannot figure out how to set one up on my development server.

There are instructions for doing this using the developers console on the live server, but I can't find anything about how to do it on my local development machine...

Answer 1

Turns out you don't need to perform any setup at all. I just assumed there was one with particular name when uploading using the blobstore and one was created for me automatically.

Incidentally, it does not seem to be documented anywhere how you can browse files in the storage of the development server. You can do it by selecting the __GsFileInfo__ entity in the Datastore Viewer admin access to your local dev server.

Answer 2

For those trying to get Google Cloud Storage to work at all from their local Java development app server, I thought one more answer would be helpful. I managed to get my local dev app server working with non-local Google Cloud Storage, but only be digging through the code and figuring out what was needed - there doesn't appear to be documentation on this.

The goal will be to get this block of code to work locally, which reads a file from GCS:

    GcsService gcsService =
       GcsServiceFactory.createGcsService(RetryParams.getDefaultInstance());
    int fileSize = (int) gcsService.getMetadata(gcsFilename).getLength();
    ByteBuffer byteBuffer = ByteBuffer.allocate(fileSize);
    GcsInputChannel inputChannel = gcsService.openReadChannel(gcsFilename, 0);
    int readResult = inputChannel.read(byteBuffer);
    byte[] fileBytes = byteBuffer.array();

If you try to do this locally, you won't find any files you've upload to GCS, because it will be trying to use a fake local GCS. Unfortunately, I haven't found a good way to upload to this local GCS, so that isn't very useful (there is no file explorer for it as there is in the cloud version, and gsutil doesn't work for it). So instead, we're going to get it to work with non-local (cloud) GCS when running in the local dev app server.

To do that, note that GcsService is created in com.google.appengine.tools.cloudstorage.GcsServiceFactory by this block of code:

if (location == SystemProperty.Environment.Value.Production || hasCustomAccessTokenProvider()) {
  rawGcsService = OauthRawGcsServiceFactory.createOauthRawGcsService(builder.build());
} else if (location == SystemProperty.Environment.Value.Development) {
  rawGcsService = LocalRawGcsServiceFactory.createLocalRawGcsService();

The above says that you need to specify a custom access token provider to get the non-local service, which you do by defining a system property. For an app engine app, you can do that in appengine-web.xml like this:

<system-properties>
    <property name="gcs_access_token_provider" value="com.mypackage.MyAccessTokenProvider" />
</system-properties>

This value of that property is a class you define that implements com.google.appengine.tools.cloudstorage.oauth.AccessTokenProvider, which provides the access token for your app. That class needs to create a GoogleCredential, which can be used to fetch an access token, using the instructions for "Other" for a GoogleCredential on https://developers.google.com/identity/protocols/OAuth2ServiceAccount#authorizingrequests.

Now it will create an OAuth GcsService that talks to the cloud, and you don't need to use the fake local storage.