kubernetes: Failed to update endpoints warning

Question

Here is something I noticed in my kubectl get events output

Warning   FailedToUpdateEndpoint   Endpoints                 Failed to update endpoint mynamespace/myservice: Operation cannot be fulfilled on endpoints "myservice": the object has been modified; please apply your changes to the latest version and try again

I am aware of this discussion, but I do not think is applicable, given I am not explicitly creating an Endpoint resource via yaml.

I am noticing some minor service unavailability during image updates so I am trying to check if this is related.

Using GKE with version v1.12.7-gke.25 on both masters and nodes, on top of istio.

Answer 1

It's common behaviour of k8s to let the k8s clients (Controllers) know to try again.

Kubernetes leverages the concept of resource versions to achieve optimistic concurrency. concurrency-control-and-consistency

It's populated by the system.

To enable clients to build a model of the current state of a cluster, all Kubernetes object resource types are required to support consistent lists and an incremental change notification feed called a watch. Every Kubernetes object has a resourceVersion field representing the version of that resource as stored in the underlying database. When retrieving a collection of resources (either namespace or cluster scoped), the response from the server will contain a resourceVersion value that can be used to initiate a watch against the server. The server will return all changes (creates, deletes, and updates) that occur after the supplied resourceVersion. This allows a client to fetch the current state and then watch for changes without missing any updates. If the client watch is disconnected they can restart a new watch from the last returned resourceVersion, or perform a new collection request and begin again efficient-detection-of-changes