Updating data from protobuf

Question

I'm building a microservice system with multiple disconnected components, and I'm currently trying to find out how to implement knowing which fields on an object should be updated based on the protobuf data provided.

The flow is this:

1. The client sends a JSON-request to an API.
2. The API translates the JSON-data into a protobuf struct, which is then sent along to the microservice responsible for handling it.
3. The microservice receives the data from the API and performs any action on it, in this case, I'm trying to change a single value in a MySQL table, such as a client's email address.

Now, the problem I have is that since protobuf (understandably) doesn't allow pointers, the protobuf object will contain zero-values for everything not provided. This means that if a customer wants to update their email address, I can't know if they also set IncludeInMailLists to false - or if it was simply not provided (having its zero-value) and shouldn't change.

The question is: how will I - from the protobuf object - know if a value is expressively set to 0, or just not provided?

My current solution is pretty much having a special UpdateCustomer-object which also has an array of Fields specifying which fields the microservice should care about, but it feels like bad solution.

Someone must have solved this better already. How should I implement it?

Answer 1

Protobufs field masks are one way.

https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.FieldMask

https://github.com/golang/protobuf/issues/225

But if you are using grpc then there's a (sort of) built in way.

Grpc wrappers

Since proto3 (protobufs v3) there's been no distinction between a primitive that is not set, and a primitive that's been set to the "zero value" (false, 0, "", etc).

Instead you can use objects or in protobufs language a "message", as objects can be nil / null. You've not mentioned what language you are working in but hopefully these examples make sense.

Given an RPC service such as:

import "google/protobuf/wrappers.proto";

service Users {
    rpc UpdateUser(UpdateUserRequest) returns (UpdateUserResponse)
}

message UpdateUserRequest {
    int32 user_id = 1;
    google.protobuf.StringValue email = 2;
}

message UpdateUserResponse {}

Note the import "google/protobuf/wrappers.proto"; is important. It given you access to the google protobufs wrappers source code here. These are not objects that have methods that allow you to test for presence.

Grpc generated code in java gives you methods such as .hasEmail() which returns true if the value is present. The getter on an unset value will still return you the zero value. I think the golang version uses pointers that you can test for nil instead of an explicit hasX() method.

More info / discussion in this github issue