Getting unencoded data from Google cloud Pub/Sub instead of base64

Question

I'm attempting to use the Python library for Pub/Sub, but I keep getting this error: TypeError: Incorrect padding. Some quick googling revealed this issue: https://github.com/GoogleCloudPlatform/google-cloud-python/pull/2527

However, this doesn't resolve the issue - in fact, printing out the data revealed that the data was not even base64 encoded at all, and setting data = raw_data in the library resolved the issue.

We're sending the message from Java, here is the code we're using:

PCollection<String> userActionsJson = userActionsRaw.apply(ParDo.of(new BigQueryRowToJson()));
       String topicNameFull = "projects/" + options.getProject() + "/topics/" +
               options.getUsersActionsTopicName() + "-" + options.getProduct();
       userActionsJson.apply(PubsubIO.Write.named("PublishToPubSub")
               .topic(topicNameFull));

however, the same thing happens attempting to send a message through the console.

Has something changed recently to mean that data is no longer base64 encoded?

Answer 1

This was an issue that has now been resolved in master: https://github.com/GoogleCloudPlatform/google-cloud-python/issues/2577