Jenkins - Unexpected executor death

Question

I see all my executors frequently changing to Dead state in one of my Jenkins slave machine(Windows 2008 R2 SP2).

Jenkins ver. 1.651.3

I have restarted Jenkins server as well as the service.

error logs-

Unexpected executor death
java.io.IOException: Failed to create a temporary file in /var/lib/jenkins/jobs/ABCD/jobs/EFGH/jobs/Build
    at hudson.util.AtomicFileWriter.<init>(AtomicFileWriter.java:68)
    at hudson.util.AtomicFileWriter.<init>(AtomicFileWriter.java:55)
    at hudson.util.TextFile.write(TextFile.java:118)
    at hudson.model.Job.saveNextBuildNumber(Job.java:293)
    at hudson.model.Job.assignBuildNumber(Job.java:351)
    at hudson.model.Run.<init>(Run.java:284)
    at hudson.model.AbstractBuild.<init>(AbstractBuild.java:167)
    at hudson.model.Build.<init>(Build.java:92)
    at hudson.model.FreeStyleBuild.<init>(FreeStyleBuild.java:34)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at jenkins.model.lazy.LazyBuildMixIn.newBuild(LazyBuildMixIn.java:175)
    at hudson.model.AbstractProject.newBuild(AbstractProject.java:1018)
    at hudson.model.AbstractProject.createExecutable(AbstractProject.java:1209)
    at hudson.model.AbstractProject.createExecutable(AbstractProject.java:144)
    at hudson.model.Executor$1.call(Executor.java:364)
    at hudson.model.Executor$1.call(Executor.java:346)
    at hudson.model.Queue._withLock(Queue.java:1365)
    at hudson.model.Queue.withLock(Queue.java:1230)
    at hudson.model.Executor.run(Executor.java:346)
Caused by: java.io.IOException: Permission denied
    at java.io.UnixFileSystem.createFileExclusively(Native Method)
    at java.io.File.createNewFile(File.java:1006)
    at java.io.File.createTempFile(File.java:1989)
    at hudson.util.AtomicFileWriter.<init>(AtomicFileWriter.java:66)
    ... 21 more

I see this error log in my slave machine

INFO: File download attempt 1
Oct 17, 2017 10:32:00 AM com.microsoft.tfs.core.clients.versioncontrol.VersionControlClient downloadFileToStreams
INFO: File download attempt 1
Oct 17, 2017 10:32:00 AM com.microsoft.tfs.core.ws.runtime.client.SOAPService executeSOAPRequestInternal
INFO: SOAP method='UpdateLocalVersion', status=200, content-length=367, server-wait=402 ms, parse=0 ms, total=402 ms, throughput=913 B/s, gzip
Oct 17, 2017 10:32:00 AM com.microsoft.tfs.core.clients.versioncontrol.VersionControlClient downloadFileToStreams
INFO: File download attempt 1
Oct 17, 2017 10:32:00 AM com.microsoft.tfs.core.clients.versioncontrol.VersionControlClient downloadFileToStreams
INFO: File download attempt 1
Oct 17, 2017 10:32:00 AM com.microsoft.tfs.core.clients.versioncontrol.VersionControlClient downloadFileToStreams
INFO: File download attempt 1

Answer 1

Can you please check the owner of the path /var/lib/jenkins/jobs/ABCD/jobs/EFGH/jobs/Build ? By any chance if it is created manually, you will get permission denied error if the owner is not Jenkins. Also check for free disk space on server as well as agent and try rebooting the slave agent. It has helped it at times.

Answer 2

How long are the real job names for ABCD and EFGH?

I've run into the 260 character maximum path length with Jenkins on Windows 2008 R2 before.