Menu
I have set up an EKS cluster using eksctl using all the default settings and now need to communicate with an external service which uses IP whitelisting. Obviously requests made to the service from my cluster come from whichever node the request was made from, but the list of nodes (and their ips) can and will change frequently so I cannot supply a single IP address for them to whitelist. After looking into this I found that I need to use a NAT Gateway.
I am having some trouble getting this to work, I have tried setting AWS_VPC_K8S_CNI_EXTERNALSNAT to true however doing so prevents all outgoing traffic on my cluster, I assume because the return packets do not know where to go so I never get the response. I've tried playing around with the route tables to no avail.
Any assistance is much appreciated.
905
The EKS node is of type m3. 2xlarge and has at least 1 pod scheduled on it. Hence, during the node attach process, the instance has 2 ENIs (active and standby) attached and 60 IP addresses allocated (2 primary IP's + 2*29 secondary IP's).
Instead of the default, EKS will only attempt to keep two additional IPs free per node. We can now see the cluster has consumed 12 IPs! That's one per node, one per pod, and two extra per node, as expected.
Worker nodes connect to the EKS control plane through the EKS public endpoint or EKS-managed elastic network interfaces (ENIs). The subnets that you pass when you create the cluster influence where EKS places these ENIs. You need to provide a minimum of two subnets in at least two Availability Zones.
You can follow this guide to create public subnets and private subnets in your VPC.
Then create NAT gateways in public subnets. Also run all EKS nodes in private subnets. The pods in K8S will use NAT gateway to access the internet services.
174
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
