Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

JSON to Protobuf in Python

Hey I know there is a solution for this in Java, I'm curious to know if anyone knows of a Python 3 solution for converting a JSON object or file into protobuf format. I would accept either or as converting to an object is trivial. Searching the stackoverflow site, I only found examples of protobuf->json, but not the other way around. There is one extremely old repo that may do this but it is in Python 2 and our pipeline is Python 3. Any help is as always, appreciated.

like image 959
jimh Avatar asked Feb 21 '20 19:02

jimh


People also ask

Does Protobuf use JSON?

It is important to note that, although JSON and Protobuf messages can be used interchangeably, these technologies were designed with different goals. JSON, which stands for JavaScript Object Notation, is simply a message format that arose from a subset of the JavaScript programming language.

What is Protobuf in Python?

Protocol buffers (Protobuf) are a language-agnostic data serialization format developed by Google. Protobuf is great for the following reasons: Low data volume: Protobuf makes use of a binary format, which is more compact than other formats such as JSON. Persistence: Protobuf serialization is backward-compatible.

Is Protobuf better than JSON?

JSON is usually easier to debug (the serialized format is human-readable) and easier to work with (no need to define message types, compile them, install additional libraries, etc.). Protobuf, on the other hand, usually compresses data better and has built-in protocol documentation via the schema.

Is Protobuf human readable?

Protobuf is a binary serialization protocol developed by Google. Since it serializes to binary, it is not human readable (although you can still pick out strings when viewing the file as ASCII text, see the example below).


1 Answers

The library you're looking for is google.protobuf.json_format. You can install it with the directions in the README here. The library is compatible with Python >= 2.7.

Example usage:

Given a protobuf message like this:

message Thing {
    string first = 1;
    bool second = 2;
    int32 third = 3;
}

You can go from Python dict or JSON string to protobuf like:

import json

from google.protobuf.json_format import Parse, ParseDict

d = {
    "first": "a string",
    "second": True,
    "third": 123456789
}

message = ParseDict(d, Thing())
# or
message = Parse(json.dumps(d), Thing())    

print(message.first)  # "a string"
print(message.second) # True
print(message.third)  # 123456789

or from protobuf to Python dict or JSON string:

from google.protobuf.json_format import MessageToDict, MessageToJson

message_as_dict = MessageToDict(message)
message_as_dict['first']  # == 'a string'
message_as_dict['second'] # == True
message_as_dict['third']  # == 123456789
# or
message_as_json_str = MessageToJson(message)

The documentation for the json_format module is here.

like image 76
kingkupps Avatar answered Sep 21 '22 01:09

kingkupps