Connection dropped for Jenkins slave on Kubernetes

Question

Performed update to Kubernetes 1.5.2 on Google Container Engine. Then started getting the following errors:

Failed to count the # of live instances on Kubernetes

To resolve this I then upgraded Jenkins (to 2.32.2) and the Kubernetes plugin (to 0.10) to the latest versions.

Afterwards, then I started getting the following errors:

Feb 08, 2017 9:51:52 PM hudson.TcpSlaveAgentListener$ConnectionHandler run
WARNING: Connection #5 failed
java.io.EOFException
    at java.io.DataInputStream.readFully(DataInputStream.java:197)
    at java.io.DataInputStream.readFully(DataInputStream.java:169)
    at hudson.TcpSlaveAgentListener$ConnectionHandler.run(TcpSlaveAgentListener.java:213)

Feb 08, 2017 9:51:57 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud$ProvisioningCallback call
SEVERE: Error in provisioning; slave=KubernetesSlave name: default-6126d6e4fb5, template=org.csanchez.jenkins.plugins.kubernetes.PodTemplate@47404ab7
java.lang.IllegalStateException: Containers are terminated with exit codes: {jnlp=255}
    at org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud$ProvisioningCallback.call(KubernetesCloud.java:600)
    at org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud$ProvisioningCallback.call(KubernetesCloud.java:532)
    at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

Answer 1

The last error was resolved by changing the slave container name to jnlp instead of default (see image). The google documentation shows the name is supposed to be default but it seems with these updates this is not the right approach to get this system working.

It looks like the updated kubernetes-plugin creates two containers (a container with your specified image and another with the default jnlp image). If your image's name isn't jnlp then the plugin will run both containers in the slave pod... this seems to be causing the connection issue.

Answer 2

As @Alex mentioned the problem is the name of the container. It has to be jnlp otherwise you get the below error.

java.lang.IllegalStateException: Pod has terminated containers: jenkins/jnlp-42t0n (prod-slave)

Initially I named the container prod-slave that caused it to create two containers in the slave pod. The additional container was named jnlp. When I renamed the container name in container template to jnlp everything worked perfectly with only one container in the slave pod.

Although it seems weird to me that naming of a container could affect the behavior of the slave!