How to pip install in a docker image with a jenkins pipline step?

Question

I have this Dockerfile:

FROM python:3.7

CMD ["/bin/bash"]

and this Jenkinsfile:

pipeline {
agent {
    dockerfile {
        filename 'Dockerfile'
    }
}
stages {
    stage('Install') {
        steps {
            sh 'pip install --upgrade pip'
        }
    }
}

This causes the following error:

The directory '/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting pip
  Downloading https://files.pythonhosted.org/packages/d8/f3/413bab4ff08e1fc4828dfc59996d721917df8e8583ea85385d51125dceff/pip-19.0.3-py2.py3-none-any.whl (1.4MB)
Installing collected packages: pip
  Found existing installation: pip 19.0.2
Uninstalling pip-19.0.2:
Could not install packages due to an EnvironmentError: [Errno 13] 
Permission denied: '/usr/local/bin/pip'
Consider using the `--user` option or check the permissions.

I have tried to user the --user, with no success.

I had some luck using args --user 0:0 on the docker jenkinsfile declaration, but this creates directories and files owned by root which can not be deleted by the user Jenkins at the next run.

I don't want to do the pip install on the Dockerfile since in reality the Install step is running a make file instead of the simplification I used above, that I want to use in other contexts.

I've also seen advice to change the HOME environment var, and this seems to fix the first 2 warnings about the parent directoy not being owned by current user, but not the Errno 13 part.

Answer 1

As I mentioned in this comment, the solution should be adding a proper user inside the container. Jenkins uses 984:984 for uid/gid on my machine (but may be different on yours - login to the host Jenkins is running on and execute sudo -u jenkins id -a to detect them), so you need to replicate it in the container that should be run by Jenkins:

FROM python:3.7

RUN mkdir /home/jenkins
RUN groupadd -g 984 jenkins
RUN useradd -r -u 984 -g jenkins -d /home/jenkins jenkins
RUN chown jenkins:jenkins /home/jenkins
USER jenkins
WORKDIR /home/jenkins

CMD ["/bin/bash"]

Of course, since you aren't the root user in the container anymore, either create a virtual environment:

$ docker run --rm -it jenkins/python /bin/bash
jenkins@d0dc87c39810:~$ python -m venv myenv
jenkins@d0dc87c39810:~$ source myenv/bin/activate
jenkins@d0dc87c39810:~$ pip install numpy

or use the --user argument:

$ docker run --rm -it jenkins/python /bin/bash
jenkins@d0dc87c39810:~$ pip install --user --upgrade pip
jenkins@d0dc87c39810:~$ pip install --user numpy

etc.

---

Alternatively, you can (but in most cases shouldn't) enter the container as root, but with jenkins group:

$ docker run --user 0:984 ...

This way, although the modified files will still change the owner, their group ownership will still be intact, so Jenkins will be able to clean up the files (or you can do it yourself, via

sh 'rm -f modified_file'

in the Jenkinsfile.

Answer 2

Using standard python:3.8 image

stage('Unit test') {
    steps {
        sh '''
            python -m venv .venv
            . .venv/bin/activate
            pip install -r requirements.txt
            pytest -v
        '''
     }
}