Python protobuf gRPC generates a dependency that doesn't exist

Question

I'm trying to create a gRPC binding for my python code via:

python -m grpc_tools.protoc -I $(pwd)/protos --python_out=./fino/pb2 --grpc_python_out=./fino/pb2 -I=$GOPATH/src -I=$GOPATH/src/github.com/gogo/protobuf/protobuf $(pwd)/protos/*

But the generated file has a dependency that doesn't exist:

from github.com.gogo.protobuf.gogoproto import gogo_pb2 as github_dot_com_dot_gogo_dot_protobuf_dot_gogoproto_dot_gogo__pb2

which is later used in:

DESCRIPTOR = _descriptor.FileDescriptor(
  name='oracle.proto',
  package='oracle',
  syntax='proto2',
  serialized_pb=_b('\n\x0coracle.proto\x12\x06oracle\x1a-github.com/gogo/protobuf/gogoproto/gogo.proto\x1a\x1fgoogle/protobuf/timestamp.proto\x1a\x1bgoogle/protobuf/empty.proto\x1a\x0btypes.proto\":\n\x0b\x41\x63\x63ountList\x12+\n\x08\x61\x63\x63ounts\x18\x01 \x03(\x0b...')
  ,
  dependencies=[github_dot_com_dot_gogo_dot_protobuf_dot_gogoproto_dot_gogo__pb2.DESCRIPTOR,google_dot_protobuf_dot_timestamp__pb2.DESCRIPTOR,google_dot_protobuf_dot_empty__pb2.DESCRIPTOR,types__pb2.DESCRIPTOR,])

Obviously, I cannot run this code. After trying to delete the non-existent import:

TypeError: Couldn't build proto file into descriptor pool!
Invalid proto descriptor for file "oracle.proto":
  oracle.proto: Import "github.com/gogo/protobuf/gogoproto/gogo.proto" has not been loaded.

I've tried adding

--include_imports --descriptor_set_out=$(pwd)/protos/all.proto  

but I'm unsure how to add it to my python files. All I want is self-contained description in my python codebase.

EDIT1: example proto file:

syntax = "proto2";
package etcdserverpb;

import "github.com/gogo/protobuf/gogoproto/gogo.proto";

option (gogoproto.marshaler_all) = true;
option (gogoproto.sizer_all) = true;
option (gogoproto.unmarshaler_all) = true;
option (gogoproto.goproto_getters_all) = false;

message Request {
    optional uint64 ID         =  1 [(gogoproto.nullable) = false];
    optional string Method     =  2 [(gogoproto.nullable) = false];
    optional string Path       =  3 [(gogoproto.nullable) = false];
    optional string Val        =  4 [(gogoproto.nullable) = false];
    optional bool   Dir        =  5 [(gogoproto.nullable) = false];
    optional string PrevValue  =  6 [(gogoproto.nullable) = false];
    optional uint64 PrevIndex  =  7 [(gogoproto.nullable) = false];
    optional bool   PrevExist  =  8 [(gogoproto.nullable) = true];
    optional int64  Expiration =  9 [(gogoproto.nullable) = false];
    optional bool   Wait       = 10 [(gogoproto.nullable) = false];
    optional uint64 Since      = 11 [(gogoproto.nullable) = false];
    optional bool   Recursive  = 12 [(gogoproto.nullable) = false];
    optional bool   Sorted     = 13 [(gogoproto.nullable) = false];
    optional bool   Quorum     = 14 [(gogoproto.nullable) = false];
    optional int64  Time       = 15 [(gogoproto.nullable) = false];
    optional bool   Stream     = 16 [(gogoproto.nullable) = false];
    optional bool   Refresh    = 17 [(gogoproto.nullable) = true];
}

message Metadata {
    optional uint64 NodeID    = 1 [(gogoproto.nullable) = false];
    optional uint64 ClusterID = 2 [(gogoproto.nullable) = false];
}

This is continuation from https://github.com/gogo/protobuf/issues/376

Answer 1

I know this is an old question, but I thought I'd give it a go. What I did to solve the problem:

1. Download the github.com/gogo/protobuf/gogoproto/gogo.proto file to your $(pwd)/protos folder; name the file gogo.proto.
2. Change the include from import "github.com/gogo/protobuf/gogoproto/gogo.proto"; to import "gogo.proto";
3. Change the command to explicitly use the proto files: python -m grpc_tools.protoc -I $(pwd)/protos --python_out=./fino/pb2 --grpc_python_out=./fino/pb2 -I=$GOPATH/src -I=$GOPATH/src/github.com/gogo/protobuf/protobuf $(pwd)/protos/gogo.proto $(pwd)/protos/metadata.proto (assuming that you named the example proto file as metadata.proto).