Providing Credentials to Google Cloud Storage API

Question

I'm trying to put together a hello world program for testing out Google Cloud Storage. My goal is to have the simplest possible program that just uploads a hard-coded file to Cloud Storage.

I've been scouring the internet for a basic tutorial, but the closest I could find is this guide for using Cloud Storage from App Engine. I've put together this program:

import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.util.ArrayList;
import java.util.Arrays;

import com.google.cloud.storage.Acl;
import com.google.cloud.storage.Acl.Role;
import com.google.cloud.storage.Acl.User;
import com.google.cloud.storage.BlobInfo;
import com.google.cloud.storage.Storage;
import com.google.cloud.storage.StorageOptions;

public class Main {
    public static void main(String[] args) throws FileNotFoundException{

        FileInputStream fileInputStream = new FileInputStream("my-file.txt");

        Storage storage = StorageOptions.getDefaultInstance().getService();

        BlobInfo blobInfo =
            storage.create(
                BlobInfo
                    .newBuilder("my-cloud-storage-bucket", "my-file.txt")
                    .setAcl(new ArrayList<>(Arrays.asList(Acl.of(User.ofAllUsers(), Role.READER))))
                    .build(),
                fileInputStream);

        String fileUrl = blobInfo.getMediaLink();

        System.out.println("Uploaded URL: " + fileUrl);
    }
}

When I run that code, I get a 401 Unauthorized error when I call storage.create(), and that's not surprising because I'm not providing a username or password. I gather that's because the example is running in App Engine, which provides the credentials that way.

I've tried passing in my credentials via a properties file:

Credential credential = new GoogleCredential.Builder()
                .setTransport(httpTransport)
                .setJsonFactory(jsonFactory)
                .setServiceAccountId("accountId")
                .setServiceAccountPrivateKeyFromP12File(
                        new File("PRIVATE_KEY_PATH"))
                .setServiceAccountScopes(scopes).build();

Storage storage = new StorageOptions.DefaultStorageFactory().create(StorageOptions.newBuilder().setCredentials(credential).build());

But that doesn't compile because the setCredentials() function needs a Credentials instance, not a Credential instance.

I've been Googling and reading through the Cloud Storage API, but I still can't figure out how I'm supposed to pass in the credentials without going through a convoluted setup process.

So, my question is: what is the simplest way to provide credentials to Google Cloud storage to upload a file?

For some background: I'm trying to compare and contrast Google Cloud Storage and Amazon S3. I can create a program like this in S3 no problem, but for some reason I'm hitting a brick wall with Cloud Storage. Definitely appreciate any insights or basic tutorials anybody can throw my way.

Answer 1

The Google Cloud client library for Java allows you to use several auth schemes, but by far the easiest is to take advantage of "Application Default Credentials." These are the credential associated with the service account that your app engine app runs as. If you use the google-cloud Java client library on App Engine or Compute Engine, the following should just work:

import static java.nio.charset.StandardCharsets.UTF_8;

import com.google.cloud.storage.Blob;
import com.google.cloud.storage.BlobId;
import com.google.cloud.storage.BlobInfo;
import com.google.cloud.storage.Storage;
import com.google.cloud.storage.StorageOptions;

Storage storage = StorageOptions.getDefaultInstance().getService();
BlobId blobId = BlobId.of("bucket", "blob_name");
BlobInfo blobInfo = BlobInfo.newBuilder(blobId).setContentType("text/plain").build();
Blob blob = storage.create(blobInfo, "Hello, Cloud Storage!".getBytes(UTF_8));

This is probably the simplest way to use auth. If you're running the program locally and not deployed to a Google environment, you can also install the gcloud command line utility and then run gcloud auth application-default login, at which point application default credentials should enable the code above to authenticate as you without any explicit authentication logic.

If you want to programmatically specify a specific service account and have the JSON key file associated with it, you can do so explicitly like so:

Storage storage = StorageOptions.newBuilder()
    .setCredentials(ServiceAccountCredentials.fromStream(
         new FileInputStream("/path/to/my/key.json")))
    .build()
    .getService();

You can also specify the path to the service account private key by using the environment variable GOOGLE_APPLICATION_CREDENTIALS. So something like:

$> GOOGLE_APPLICATION_CREDENTIALS=/path/to/key.json java MyApp

Answer 2

It seems it's not so easy to create credentials from a PKCS #12 file with new Google Cloud Client Library as it used to be with the old Cloud Storage JSON API.

The easiest way would be to use JSON format instead as described here, and then use GoogleCredentials#fromStream method to load it:

Credentials credentials = GoogleCredentials.fromStream(new FileInputStream("credentials.json"));
Storage storage = StorageOptions.newBuilder().setCredentials(credentials).build().getService();

If you still want to use PKCS #12, I believe this would work:

PrivateKey privateKey = SecurityUtils.loadPrivateKeyFromKeyStore(
    SecurityUtils.getPkcs12KeyStore(), new FileInputStream("credentials.p12"), "notasecret", "privatekey", "notasecret");
Credentials credentials = new ServiceAccountCredentials(null, "accountId", privateKey, null, null);
Storage storage = StorageOptions.newBuilder().setCredentials(credentials).build().getService();