How come my kubernetes' service can't find an endpoint?

Question

I am running a kubernetes cluster on coreos.

I have a kubernetes replication controller that works fine. It looks like this:

id: "redis-controller"
kind: "ReplicationController"
apiVersion: "v1beta3"
metadata:
  name: "rediscontroller"
  lables:
    name: "rediscontroller"
spec:
  replicas: 1
  selector:
    name: "rediscontroller"
  template:
    metadata:
      labels:
        name: "rediscontroller"
    spec:
      containers:
        - name: "rediscontroller"
          image: "redis:3.0.2"
          ports:
            - name: "redisport"
              hostPort: 6379
              containerPort:  6379
              protocol: "TCP"

But I have a service for said replication controller's pods that looks like this:

id: "redis-service"
kind: "Service"
apiVersion: "v1beta3"
metadata:
  name: "redisservice"
spec:
  ports:
    - protocol: "TCP"
      port: 6379
      targetPort: 6379
  selector:
    name: "redissrv"
  createExternalLoadBalancer: true
  sessionAffinity: "ClientIP"

the journal for kube-proxy has this to say about the service:

Jul 06 21:18:31 core-01 kube-proxy[6896]: E0706 21:18:31.477535    6896 proxysocket.go:126] Failed to connect to balancer: failed to connect to an endpoint.
Jul 06 21:18:41 core-01 kube-proxy[6896]: E0706 21:18:41.353425    6896 proxysocket.go:81] Couldn't find an endpoint for default/redisservice:: missing service entry

From what I understand, I do have the service pointing at the right pod and right ports, but am I wrong?

UPDATE 1

I noticed another possible issue, after fixing the things mentioned by Alex, I noticed in other services, where it is using websockets, the service can't find an endpoint. Does this mean the service needs a http endpoint to poll?

Answer 1

Extra thing to check for.

Endpoints are only created if your deployment is considered healthy. If you have defined your readinessProbe incorrectly (mea culpa) or the deployment does not react to it correctly, an endpoint will not be created.

Answer 2

You can try inspecting the endpoints with kubectl get ep kubectl describe ep. If you see pod IP's next to NotReadyAddresses in the endpoints description, this indicates there's a problem with the pod that's causing it not to be ready, in which case it will fail to be registered against the endpoints.

If the pod isn't ready it can be because of a failing health/liveness probe.

The 'selector' on your service (kubectl get services kubectl describe myServiceName) should match a label on the pods (kubectl get pods kubectl describe po myPodName). E.g. selector = app=myAppName, pod label = app=myAppName. That's how the service determines which of the endpoints it should be trying to connect to.

Answer 3

A few things look funny to me, with the first two being most important:

1. It looks like the service doesn't exist. Are you sure it was created properly? Does it show up when you run kubectl get svc?
2. The selector on your service doesn't look right. The selector should be key-value label pairs that match those in the replication controller's template. The label in your rc template is name: "rediscontroller", so you should use that as your service selector as well.
3. What's the id field at the start of each object? It doesn't look like that's a valid field in v1beta3.

Answer 4

For you particular case, make sure the service spec has a containerPort if you specified it in your Pod spec. See details: http://kubernetes.io/docs/user-guide/debugging-services/#my-service-is-missing-endpoints

Otherwise please setup through the official K8s service debugging guide:

http://kubernetes.io/docs/user-guide/debugging-services/

It has a step-by-step checklist of things to look out to from service to DNS to networking to kube proxy etc.