Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to allow access to kubernetes api using egress network policy?

Tags:

Init container with kubectl get pod command is used to get ready status of other pod.

After Egress NetworkPolicy was turned on init container can't access Kubernetes API: Unable to connect to the server: dial tcp 10.96.0.1:443: i/o timeout. CNI is Calico.

Several rules were tried but none of them are working (service and master host IPs, different CIDR masks):

...
  egress:
  - to:
    - ipBlock:
        cidr: 10.96.0.1/32
    ports:
    - protocol: TCP
      port: 443
...

or using namespace (default and kube-system namespaces):

...
  egress:
  - to:
    - namespaceSelector:
        matchLabels:
          name: default
    ports:
    - protocol: TCP
      port: 443
...

Looks like ipBlock rules just don't work and namespace rules don't work because kubernetes api is non-standard pod.

Can it be configured? Kubernetes is 1.9.5, Calico is 3.1.1.

Problem still exists with GKE 1.13.7-gke.8 and calico 3.2.7

like image 513
Igor Stepin Avatar asked Apr 30 '18 14:04

Igor Stepin


People also ask

How do I enable Kubernetes API?

This page shows how to enable or disable an API version from your cluster's control plane. Specific API versions can be turned on or off by passing --runtime-config=api/<version> as a command line argument to the API server.

How does egress work in Kubernetes?

In the context of Kubernetes egress, NAT is used to allow pods to connect to services outside of the cluster if the pods have IP addresses that are not routable outside of the cluster (for example, if the pod network is an overlay).

Can we specify egress rules with network policies?

Network policies can be used to specify both allowed ingress to pods and allowed egress from pods. These specifications work as one would expect: traffic to a pod from an external network endpoint outside the cluster is allowed if ingress from that endpoint is allowed to the pod.

What is ingress and egress Kubernetes?

Ingress and egress From the point of view of a Kubernetes pod, ingress is incoming traffic to the pod, and egress is outgoing traffic from the pod. In Kubernetes network policy, you create ingress and egress “allow” rules independently (egress, ingress, or both).


2 Answers

You need to get the real ip of the master using kubectl get endpoints --namespace default kubernetes and make an egress policy to allow that.

---
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1 
metadata:
  name: allow-apiserver
  namespace: test
spec:
  policyTypes:
  - Egress
  podSelector: {}
  egress:
  - ports:
    - port: 443
      protocol: TCP
    to:
    - ipBlock:
        cidr: x.x.x.x/32
like image 198
Dave McNeill Avatar answered Sep 20 '22 15:09

Dave McNeill


Update: Try Dave McNeill's answer first.

If it does not work for you (it did for me!), the following might be a workaround:

  podSelector:
    matchLabels:
      white: listed
  egress:
    - to:
        - ipBlock:
            cidr: 0.0.0.0/0

This will allow accessing the API server - along with all other IP addresses on the internet :-/

You can combine this with the DENY all non-whitelisted traffic from a namespace rule to deny egress for all other pods.

like image 35
schnatterer Avatar answered Sep 21 '22 15:09

schnatterer