How to use Firebase Storage image with Google Cloud Vision API?

Question

I have a webapp where users are authenticated anonymously with Firebase Auth. I store images from users in Firebase Storage, which behind the scenes is backed by a Google Cloud Storage bucket. When I try to use the Google Cloud Vision API from client-side javascript to get the image properties I get a permission error.

image-annotator::User lacks permission.: Can not open file: gs://MYAPP.appspot.com/photos/IMG_12345.jpg

If I make the image public, everything works. But this is user data and can't be public. How can I solve this?

My code for calling the vision api:

gapi.client.init({
    'apiKey': MY_API_KEY,
    'discoveryDocs': ['https://vision.googleapis.com/$discovery/rest'],
    // clientId and scope are optional if auth is not required.
    'clientId': MY_APP_ID + '.apps.googleusercontent.com',
    'scope': 'profile',
  }).then(function() {
    // 3. Initialize and make the API request.
    return gapi.client.vision.images.annotate({
      "requests":
      [
        {
          "features":
          [
            {
              "type": "LABEL_DETECTION"
            },
            {
              "type": "IMAGE_PROPERTIES"
            }
          ],
          "image":
          {
            "source":
            {
              "gcsImageUri": "gs://MY_APP.appspot.com/photos/" + "IMG_12345.jpg"
            }
          }
        }
      ]
    });
  }).then(function(response) {
    console.log(response.result);
  }, function(reason) {
    console.log('Error: ' + reason.result.error.message);
  });

My code for uploading images to storage:

var file = e.target.files[0];
var metadata = {
  contentType: file.type
};
console.log(file);

var uploadTask = photosStorageRef.child(file.name).put(file, metadata);

Answer 1

You really want to do this on the server side via something like Google Cloud Functions (see my example here).

Client side will require that your API key have sufficient permission to access the photos, which will basically make the private photos public (because once the API key has permission to read them, what's to stop a malicious client from doing so?).