Unhealthy nodes for load balancer when using nginx ingress controller on GKE

Question

I have set up the nginx ingress controller following this guide.

The ingress works well and I am able to visit the defaultbackend service and my own service as well.

But when reviewing the objects created in the Google Cloud Console, in particular the load balancer object which was created automatically, I noticed that the health check for the other nodes are failing:

Is this because the ingress controller process is only running on the one node, and so it's the only one that passes the health check? How do I make the other nodes pass?

Answer 1

Your assumption is correct. The healthy node is indeed the one running the nginx pod.

The guide you're using configures the service with externalTrafficPolicy: Local.(https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/gce-gke/service.yaml)

That policy configures kube-proxy to never route traffic for a service off of the node where it arrives. So, when the load balancer traffic reached the nodes that have no nginx pod, the health check failed and the load balancer stopped sending traffic to them.

This configuration had the advantage of avoiding an extra network hop to get to the nginx pod. If you need more nodes to handle the traffic, you can ensure that there are nginx pods running there too. If you don't mind the extra network hop, you can change the externalTrafficPolicy too.