How are protobuf packages used?

Question

I don't understand the first part of the last sentence on packages from google's Python protobuf docs:

The .proto file starts with a package declaration, which helps to prevent naming conflicts between different projects. In Python, packages are normally determined by directory structure, so the package you define in your .proto file will have no effect on the generated code. However, you should still declare one to avoid name collisions in the Protocol Buffers name space as well as in non-Python languages.

It's actually false that "the package you define in your .proto file will have no effect on the generated code" in Python (and presumably therefore in languages which do use namespaces) because the generated ..._pb2.py files contain descriptors which have a package field.

DESCRIPTOR = _descriptor.FileDescriptor(
  name='example.proto',
  package='example',
  syntax='proto3',
   ...
)

I have checked and you cannot import example into a regular python file after having generated the code. i.e. declaring package example; in your .proto file and then protoc compiling it doesn't automatically make your package available for import in python files in your environment (you must still use the import ..._pb2 to access the classes).

Is anyone able to explain how packages declared in .proto files are used both in Python and say C# as an example of a language which makes explicit use of namespaces?

Answer 1

Quoting from the documentation itself:

In Python, the package directive is ignored, since Python modules are organized according to their location in the file system.

So what exactly is the use of packages declaration then? Two things I believe:

1. To avoid the possibility of naming conflicts in the message objects. If supposedly you have a message which has been declared differently under the same application, then you can consider providing the package specifier at the beginning of the proto file, and then use the package specified when defining fields of your other message types. For example, if Foo is defined under package bar and package baz both; then use bar.Foo and baz.Foo wherever required.

2. For portability. Package directive is ignored both in Python and Golang. But languages like C++ and C# which create namespaces explicitly, do consider the package specified. So for Foo in the above example, a message type named Foo would be declared both in namespaces of bar and baz. The proto file you created with your Python application can be compiled for your C# application in much the same way. It is to be noted that Protobuf is language agnostic mechanism for serializing and deserializing data.

Having said that, I haven't personally much found the use of the package specifier in my protobuf message with my Python applications.

Ref: https://developers.google.com/protocol-buffers/docs/proto