Kubernetes Jenkins plugin - slaves always offline

Question

I'm trying to use the Jenkins/Kubernetes plugin to orchestrate docker slaves with Jenkins.

I'm using this plugin: https://github.com/jenkinsci/kubernetes-plugin

My problem is that all the slaves are offline so the job can't execute:

I have tried this on my local box using minikube, and on a K8 Cluster hosted by our ops group. I've tried both Jenkins 1.9 and Jenkins 2. I always get the same result. The screenshots are from Jenkins 1.642.4, K8 v1.2.0

Here is my configuration... note that when I click 'test connection' I get a success. Also note I didn't need any credentials (this is the only difference I can see vs the documented example).

The Jenkins log shows the following over and over:

    Waiting for slave to connect (11/100): docker-6b55f1b7fafce
Jul 20, 2016 5:01:06 PM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud$ProvisioningCallback call
Waiting for slave to connect (12/100): docker-6b55f1b7fafce
Jul 20, 2016 5:01:07 PM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud$ProvisioningCallback call
Waiting for slave to connect (13/100): docker-6b55f1b7fafce
Jul 20, 2016 5:01:08 PM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud$ProvisioningCallback call

When I run kubectl get events I see this:

24s         24s        1         docker-6b3c2ff27dad3   Pod                                Normal    Scheduled           {default-scheduler }      Successfully assigned docker-6b3c2ff27dad3 to 96.xxx.xx.159
24s         23s        2         docker-6b3c2ff27dad3   Pod                                Warning   MissingClusterDNS   {kubelet 96.xxx.xx.159}   kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Falling back to DNSDefault policy.
23s         23s        1         docker-6b3c2ff27dad3   Pod       spec.containers{slave}   Normal    Pulled              {kubelet 96.xxx.xx.159}   Container image "jenkinsci/jnlp-slave" already present on machine
23s         23s        1         docker-6b3c2ff27dad3   Pod       spec.containers{slave}   Normal    Created             {kubelet 96.xxx.xx.159}   Created container with docker id 82fcf1bd0328
23s         23s        1         docker-6b3c2ff27dad3   Pod       spec.containers{slave}   Normal    Started             {kubelet 96.xxx.xx.159}   Started container with docker id 82fcf1bd0328

Any ideas?

UPDATE: more log info as suggested by csanchez

 ➜  docker git:(master) ✗ kubectl get pods --namespace default -o wide
NAME                   READY     STATUS    RESTARTS   AGE       NODE
docker-6bb647254a2a4   1/1       Running   0          1m        96.x.x.159

➜  docker git:(master) ✗ kubectl log docker-6bafbac10b392

    Jul 20, 2016 6:45:10 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to 96.x.x.159:50000 (retrying:10)
java.net.ConnectException: Connection refused
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)

I'll have to look at what this port 50000 is used for??

Answer 1

Because, kubectl -n jenkins-namespace get services shows :

NAME                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)     AGE
cicd-jenkins         ClusterIP   172.20.120.227   <none>        8080/TCP    128d
cicd-jenkins-agent   ClusterIP   172.20.105.189   <none>        50000/TCP   128d

You have to go to Jenkins > Manage Jenkins > Configure System (http://jenkins:8080/configure). Then, configure Jenkins URL and jenkins tunnel accordingly (see screenshot below )

CREDITS to https://youtu.be/MkzCVvlpiaM

If you'are using the Jenkins Configuration as Code (JCasC) plugin this is configured via jenkinsUrl and jenkinsTunnel keys :

jenkins:
  clouds:
  - kubernetes:
      name: cluster
      serverUrl: https://kubernetes.default
      # ....
      jenkinsUrl: http://cicd-jenkins:8080/
      jenkinsTunnel:  cicd-jenkins-agent:50000
      # ....