I would like to release a python package for a set of protobuf messages. The protobuf compiler (protoc
) generates a python library that does not actually define types/classes the typical sense, but rather dynamically constructs them. Is there any way to hint to pylint what the members and fields of these classes are?
For example, consider the following simple protobuf message specification:
message Person {
required string name = 1;
required int32 id = 2;
optional string email = 3;
}
The compiler generates the following long pile of code:
# Generated by the protocol buffer compiler. DO NOT EDIT!
# source: test.proto
import sys
_b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
from google.protobuf import descriptor as _descriptor
from google.protobuf import message as _message
from google.protobuf import reflection as _reflection
from google.protobuf import symbol_database as _symbol_database
from google.protobuf import descriptor_pb2
# @@protoc_insertion_point(imports)
_sym_db = _symbol_database.Default()
DESCRIPTOR = _descriptor.FileDescriptor(
name='test.proto',
package='',
serialized_pb=_b('\n\ntest.proto\"1\n\x06Person\x12\x0c\n\x04name\x18\x01 \x02(\t\x12\n\n\x02id\x18\x02 \x02(\x05\x12\r\n\x05\x65mail\x18\x03 \x01(\t')
)
_sym_db.RegisterFileDescriptor(DESCRIPTOR)
_PERSON = _descriptor.Descriptor(
name='Person',
full_name='Person',
filename=None,
file=DESCRIPTOR,
containing_type=None,
fields=[
_descriptor.FieldDescriptor(
name='name', full_name='Person.name', index=0,
number=1, type=9, cpp_type=9, label=2,
has_default_value=False, default_value=_b("").decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
options=None),
_descriptor.FieldDescriptor(
name='id', full_name='Person.id', index=1,
number=2, type=5, cpp_type=1, label=2,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
options=None),
_descriptor.FieldDescriptor(
name='email', full_name='Person.email', index=2,
number=3, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=_b("").decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
options=None),
],
extensions=[
],
nested_types=[],
enum_types=[
],
options=None,
is_extendable=False,
extension_ranges=[],
oneofs=[
],
serialized_start=14,
serialized_end=63,
)
DESCRIPTOR.message_types_by_name['Person'] = _PERSON
Person = _reflection.GeneratedProtocolMessageType('Person', (_message.Message,), dict(
DESCRIPTOR = _PERSON,
__module__ = 'test_pb2'
# @@protoc_insertion_point(class_scope:Person)
))
_sym_db.RegisterMessage(Person)
# @@protoc_insertion_point(module_scope)
import sys import myprotocol_pb2 as proto import varint # (this is the varint.py file) data = open("filename. bin", "rb"). read() # read file as string decoder = varint. decodeVarint32 # get a varint32 decoder # others are available in varint.py next_pos, pos = 0, 0 while pos < len(data): msg = proto.
Protocol Buffers (Protobuf) is a free and open-source cross-platform data format used to serialize structured data. It is useful in developing programs to communicate with each other over a network or for storing data.
The Protobuf is a binary transfer format, meaning the data is transmitted as a binary. This improves the speed of transmission more than the raw string because it takes less space and bandwidth. Since the data is compressed, the CPU usage will also be less.
Protocol buffers are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once … The ProtoBuf interface describes the structure of the data to be sent.
There didn't seem to be any support for automatically hinting to Pylint the names of the fields on protobuf messages, so I cobbled together a small extension to Pylint to statically determine these.
Introducing pylint-protobuf
:
from example_pb2 import Person
p = Person()
p.invalid_field = 123
Usage:
$ pip install pylint-protobuf
$ pylint --load-plugins=pylint_protobuf example.py
************* Module example
E: 3, 0: Field 'invalid_field' does not appear in the declared
fields of protobuf-generated class 'Person' and will raise
AttributeError on access (protobuf-undefined-attribute)
Fair warning: it's pretty alpha, features not supported (yet!) include nested imports and repeated fields.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With