Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Dynamically creating python class from a protobuf file at run time?

In Python, is it possible at run time to convert a Google Protocol Buffers .proto file into a python class that reads that data? Python is a very dynamic language. When you use protoc to convert a .proto file to python source code, the generated code makes a lot of use of python metaclasses, so it's already very dynamic.

Ideally, I'm thinking of something like this:

import whatever
module = whatever.load_from_file("myfile.proto")

Is this possible?

(I am new to protocol buffers, please let me know if my question makes no sense)

like image 455
Amandasaurus Avatar asked Feb 11 '16 14:02

Amandasaurus


People also ask

How to compile Protobuf files to Python objects?

To compile proto files to Python objects, we are going to use the Protobuf compiler, protoc. We will call the proto compiler with the following options: --proto_path: because the proto files are not in the root folder of the project, we need to use a substitution.

Is it possible to use Protobuf with C++ metaclasses?

In theory, all the pieces exist to make this work. The Python protobuf implementation could call the C++ .proto parser library ( libprotoc) as a C extension to get Descriptors, and then could feed those into the metaclasses. However, as far as I know, no one has quite tied it altogether.

How to automatically compile Proto files upon installation of a Python package?

To automatically compile the proto files upon installation of a development Python package, we can use the setup.py script. Let’s create a function that generates the Protobuf code for all .proto files in the src/interfaces directory and stores them under src/generated:

Is it possible to convert a protoc file to Python?

Python is a very dynamic language. When you use protoc to convert a .proto file to python source code, the generated code makes a lot of use of python metaclasses, so it's already very dynamic. Ideally, I'm thinking of something like this:


1 Answers

In theory, all the pieces exist to make this work. The Python protobuf implementation could call the C++ .proto parser library (libprotoc) as a C extension to get Descriptors, and then could feed those into the metaclasses.

However, as far as I know, no one has quite tied it altogether. (Disclaimer: My knowledge is a few years old, I may have missed a new development, but I don't see anything in the docs.)

Incidentally, Cap'n Proto's Python implementation does do what you describe, proving it is possible. But that doesn't help you if you need to work with Protobuf format.

(Disclosure: I was the author of most of Google's open source Protobuf code, and I am also the author of Cap'n Proto.)

like image 75
Kenton Varda Avatar answered Sep 16 '22 15:09

Kenton Varda