Jenkins: dockerfile agent commands not run in container

Question

This is my Jenkinsfile:

pipeline {
    agent {
        dockerfile true
    }

    stages {
        stage('Run tests') {
            steps {
                sh 'pwd'
                sh 'vendor/bin/phpunit'
            }
        }
    }
}

I'm running Jenkins, and although I am able to build the image successfully, "Run tests" is run outside of the new container in the host. This is not good; I want the command to run from within the new container built with the help of the dockerfile agent.

I know that the shell command is run in the host, because I've already tried debugging with sh pwd to which I got /var/jenkins_home/workspace/youtube-delete-tracker_jenkins.

Here is the end of the output in the console for the Jenkins job:

Step 18/18 : RUN chmod a+rw database/
 ---> Using cache
 ---> 0fedd44ea512
Successfully built 0fedd44ea512
Successfully tagged e74bf5ee4aa59afc2c4524c57a81bdff8a341140:latest
[Pipeline] dockerFingerprintFrom
[Pipeline] }
[Pipeline] // stage
[Pipeline] sh
+ docker inspect -f . e74bf5ee4aa59afc2c4524c57a81bdff8a341140
.
[Pipeline] withDockerContainer
Jenkins does not seem to be running inside a container
$ docker run -t -d -u 112:116 -w /var/jenkins_home/workspace/ube-delete-tracker_stackoverflow -v /var/jenkins_home/workspace/ube-delete-tracker_stackoverflow:/var/jenkins_home/workspace/ube-delete-tracker_stackoverflow:rw,z -v /var/jenkins_home/workspace/ube-delete-tracker_stackoverflow@tmp:/var/jenkins_home/workspace/ube-delete-tracker_stackoverflow@tmp:rw,z -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** e74bf5ee4aa59afc2c4524c57a81bdff8a341140 cat
$ docker top 64bbdf257492046835d7cfc17413fb2d78c858234aa5936d7427721f0038742b -eo pid,comm
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Run tests)
[Pipeline] sh
+ pwd
/var/jenkins_home/workspace/ube-delete-tracker_stackoverflow
[Pipeline] sh
+ vendor/bin/phpunit
/var/jenkins_home/workspace/ube-delete-tracker_stackoverflow@tmp/durable-4049973d/script.sh: 1: /var/jenkins_home/workspace/ube-delete-tracker_stackoverflow@tmp/durable-4049973d/script.sh: vendor/bin/phpunit: not found
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
$ docker stop --time=1 64bbdf257492046835d7cfc17413fb2d78c858234aa5936d7427721f0038742b
$ docker rm -f 64bbdf257492046835d7cfc17413fb2d78c858234aa5936d7427721f0038742b
[Pipeline] // withDockerContainer
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline

As you can see, pwd gives me a path on the host (the Jenkins job folder), and vendor/bin/phpunit was not found (which should be there in the container, because the package manager successfully built it as per the docker build output that I didn't include).

So how can I get the sh commands running from within the container? Or alternatively how do I get the image tag name generated by the dockerfile agent so that I could manually do docker run from outside the new container to run the new container?

INFO: The issue doesn't seem to have to do with the Declarative Pipelines, because I just tried doing it Imperative style and also get the same pwd of the Jenkins container: https://github.com/amcsi/youtube-delete-tracker/blob/4bf584a358c9fecf02bc239469355a2db5816905/Jenkinsfile.groovy#L6

INFO 2: At first I thought this was an Jenkins-within-Docker issue, and I wrote my question as such... but it turned out I was getting the same issue if I ran Jenkins on my host rather than within a container.

INFO 3: Versions...

Jenkins ver. 2.150.1
Docker version 18.09.0, build 4d60db4
Ubuntu 18.04.1 LTS

Answer 1

EDIT:

Jenkins mounts its local workspace into the docker container, and cd into it automatically. Therefore, the WORKDIR you set in Dockerfile gets over-ridden. In your console output, the docker run command shows this fact:

docker run -t -d -u 112:116 -w /var/jenkins_home/workspace/ube-delete-tracker_stackoverflow -v /var/jenkins_home/workspace/ube-delete-tracker_stackoverflow:/var/jenkins_home/workspace/ube-delete-tracker_stackoverflow:rw,z

From docker run man page (man docker run):

   -w, --workdir=""
      Working directory inside the container

The default working directory for running binaries within a container is the root directory (/). The developer can set a different default with the Dockerfile WORKDIR instruction. The operator can override the working directory by using the -w option.

   -v|--volume[=[[HOST-DIR:]CONTAINER-DIR[:OPTIONS]]]
      Create a bind mount. 

If you specify, -v /HOST-DIR:/CONTAINER-DIR, Docker bind mounts /HOST-DIR in the host to /CONTAINER-DIR in the Docker container. If 'HOST-DIR' is omitted, Docker automatically creates the new volume on the host. The OPTIONS are a comma delimited list and can be:

So, you need to manually cd into your own $WORKDIR first before executing the said commands. Btw, you might also want to create a symbolic link (ln -s) from jenkins volume mount dir to your own $WORKDIR. This allows you to browse the workspace via the Jenkins UI etc.

Finally, to re-iterate for clarity, Jenkins run all its build stages inside a docker container if you specify the docker agent at the top.

---

Old answer:

Jenkins run all its build stages inside a docker container if you specify the docker agent at the top. You can verify under which executor and slave the build is running on using environment variables.

So, first find out an environment variable that is specific to your slave. I use NODE_NAME env var for this purpose. You can find out all available environment variables in your Jenkins instance via http://localhost:8080/env-vars.html (replace the host name to match your instance).

...
stages {
    stage('Run tests') {
        steps {
            echo "I'm executing in node: ${env.NODE_NAME}"
            sh 'vendor /bin/phpunit'
        }
    }
}
...

A note regarding vendor/bin/phpunit was not found - this seems to be due to a typo if I'm not mistaken.