Google cloud vision not accepting base64 encoded images python

Question

I'm having a problem with base64 encoded images sent to Google Cloud Vision. Funny thing is that if I send the image via URI, it works fine, so I suspect there is something wrong the way I'm encoding.

Here's the deal:

from google.cloud import vision
import base64
client = vision.ImageAnnotatorClient()
image_path ='8720911950_91828a2aeb_b.jpg'
with open(image_path, 'rb') as image:
    image_content = image.read()
    content = base64.b64encode(image_content)   
    response = client.annotate_image({'image': {'content': content}, 'features': [{'type': vision.enums.Feature.Type.LABEL_DETECTION}],})
    print(response)

The response I get always is:

error {
  code: 3
  message: "Bad image data."
}

If I try using URI instead:

response = client.annotate_image({'image': {'source': {'image_uri': 'https://farm8.staticflickr.com/7408/8720911950_91828a2aeb_b.jpg'}}, 'features': [{'type': vision.enums.Feature.Type.LABEL_DETECTION}],})

Response is ok...

label_annotations {
  mid: "/m/0168g6"
  description: "factory"
  score: 0.7942917943000793
}
label_annotations {
  mid: "/m/03rnh"
  description: "industry"
  score: 0.7761002779006958
}

I've followed the recommended way to encode from Google

Any idea what is wrong here?

Answer 1

I don't have any experience with Google Cloud Vision, however after looking at their documentation and examples, my feeling is that the linked documentation page about base64 encoding of image data is for the case when you create and send the HTTP requests on your own, without using vision.ImageAnnotatorClient. The latter seems to encode the image data automatically, hence in your example double encoding is applied. Therefore I believe that you should remove the encoding step from your code:

from google.cloud import vision
import base64
client = vision.ImageAnnotatorClient()
image_path ='8720911950_91828a2aeb_b.jpg'
with open(image_path, 'rb') as image:
    content = image.read()
    response = client.annotate_image({'image': {'content': content}, 'features': [{'type': vision.enums.Feature.Type.LABEL_DETECTION}],})
    print(response)

Answer 2

Well, if you still want to use base64 encoded image data, you will have to convert it into byte array using module before sending request to annotate image. This base64 to bytearray should be used when creating API or when you are receiving input in the form of encoded data without actual path/url. Otherwise, use as it is by providing path or url as pointed out by @Leon.

import binascii
content = binascii.a2b_base64(base64_encoded_image_data)

pass this content as value for content argument in annotate_image method. Then, you will get a correct response.