Permission denied error invoking Docker on Mac host from inside Docker Ubuntu container as non-root user

Question

I'm trying to invoke docker on my OSX host running Docker for Mac 17.06.0-ce-mac17 from inside a running jenkins docker container (jenkins:latest), per the procedure described at http://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/.  

I mount /var/run/docker.sock into the container, I stick a ubuntu docker binary inside it, and it's able to execute - but from inside the container as user "jenkins" when I run e.g. "docker ps" I get

Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.30/containers/json?all=1: dial unix /var/run/docker.sock: connect: permission denied.  

If I connect to the container as root (docker exec -u 0) it works though.

I need the jenkins user to be able to run this. I tried adding a docker group and adding jenkins to it inside the ubuntu container but that didn't help, since it's got nothing to do with the outside and Docker for Mac doesn't work like running this on linux where you can do semi easy uid/gid matching. I want to distribute this container so answers that go and hack part of my Docker for Mac install won't really help me. I'd rather not run the whole jenkins setup as root if I can help it. (I also tried running the container as privileged, that didn't help.)

Per the advice in Permission Denied while trying to connect to Docker Daemon while running Jenkins pipeline in Macbook I chowned the /var/run/docker.sock file inside the container manually to jenkins and now jenkins can run docker. But I'm having trouble coming up with a solution for a distributable container - I can't do that chown in the Dockerfile because the file doesn't exist yet, and shimming in into the entrypoint doesn't help because that runs as jenkins.

What do I need to do in order to build and run an image that will run external docker containers on my Mac as a non-root user from inside the container?

Answer 1

I got this working, at least automated but currently only working on docker for Mac. Docker for Mac has a unique file permission model. Chowning /var/run/docker.sock to the jenkins user manually works, and it persists across container restarts and even image regeneration, but not past docker daemon restarts. Plus, you can't do the chown in the Dockerfile because docker.sock doesn't exist yet, and you can't do it in the entrypoint because that runs as jenkins.

So what I did was add jenkins to the "staff" group, because on my Mac, /var/run/docker.sock is symlinked down into /Users//Library/Containers/com.docker.docker/Data/‌​s60 and is uid and gid staff. This lets the jenkins user run docker commands on the host.

Dockerfile:

FROM jenkins:latest

USER root

RUN \
    apt-get update && \
    apt-get install -y build-essential && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

COPY docker /usr/bin/docker

# To allow us to access /var/run/docker.sock on the Mac
RUN gpasswd -a jenkins staff

USER jenkins

ENTRYPOINT ["/bin/tini", "--", "/usr/local/bin/jenkins.sh"]

docker-compose.yml file:

version: "3"
services:
  jenkins:
    build: ./cd_jenkins
    image: cd_jenkins:latest
    ports:
      - "8080:8080"
      - "5000:5000"
    volumes:
      - ./jenkins_home:/var/jenkins_home
      - /var/run/docker.sock:/var/run/docker.sock

This is, however, not portable to other systems (and depends on that docker for mac group staying "staff," which I imagine isn't guaranteed). I'd love suggested improvements to make this solution work across host systems. Other options suggested in questions like Execute docker host command inside jenkins docker container include:

• Install sudo and let jenkins sudo and run all docker commands with sudo: adds security issues
• "Add jenkins to the docker group" - UNIX only and probably relies on matching up gids from host to container right?
• Setuid'ing the included docker executable might work, but has the same security elevation issues as sudo.

Answer 2

Another approach that worked for me - set the uid argument to the uid that owns /var/run/docker.sock (501 in my case). Not sure of the syntax for Dockerfile, but for docker-compose.yml, it's like this:

version: 3
services:
  jenkins:
    build:
      context: ./JENKINS
      dockerfile: Dockerfile
      args:
        uid: 501
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    ...

Note this is based on using a Dockerfile to build the jenkins image, so many details left out. The key bit here is the uid: 501 under args.