Kubernetes update changes static+reserved external IPs for nodes in Google Cloud

Question

I have three nodes in my google container cluster.

Everytime i perform a kubernetes update through the web-ui on the cluster in Google Container Engine.

My external IP's change, and i have to manually assign the previous IP on all three instances in Google Cloud Console.

These are reserved static external IP set up using the following guide.

Reserving a static external IP

Has anyone run into the same problem? Starting to think this is a bug.

Perhaps you can set up the same static outbound external IP for all the instances to use, but i cannot find any information on how to do so, that would be a solution as long as it persists through updates, otherwise we've got the same issue.

It's only updates that cause this, not restarts.

Answer 1

I was having the same problem as you. We found some solutions.

• KubeIP - But this needed a cluster 1.10 or higher. Ours is 1.8
• NAT - At GCP documentation they talk about this method. It was too complex for me.

Our Solution We followed the documentation for assign IP addresses on GCE. Used the command line. Using this method, we didn't have any problems so far. I don't know the risks for it yet. If anyone has an idea, it would be good.

We basically just ran:

gcloud compute instances delete-access-config [INSTANCE_NAME]  --access-config-name [CONFIG_NAME]
gcloud compute instances add-access-config [INSTANCE_NAME] --access-config-name "external-nat-static" --address [IP_ADDRESS]

If anyone have any feedback on this solution. Please give it to us. @Ahmet Alp Balkan - Google

Answer 2

You should not rely on the IP addresses of each individual node. Instances can come and go (especially when you use Cluster Autoscaler), and their IP addresses can change.

You should always be exposing your applications with Service or Ingress and IP addresses of the load balancers created with these resources do not change between upgrades. Further you can convert IP address on a load balancer to a static (reserved) IP address.

I see that you're assigning static IP addresses to your nodes. I don't see any reason to do that. When you expose your services with Service/Ingress resources, you can associate a static external IP to them.

See this tutorial: https://cloud.google.com/container-engine/docs/tutorials/http-balancer#step_5_optional_configuring_a_static_ip_address