Docker is not running on Colab

Question

I have tried to install Docker on google Colab through the following ways:

(1)https://phoenixnap.com/kb/how-to-install-docker-on-ubuntu-18-04

(2)https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-18-04

(3)https://colab.research.google.com/drive/10OinT5ZNGtdLLQ9K399jlKgNgidxUbGP

I started the docker service and saw the status, but it showed 'Docker is not running'. Maybe the docker can not work on the Colab.

I feel confused and want to know the reason.

Thanks

Answer 1

I had the same issue as you and apparently Docker is not supported in Google Colab according to the answers on this issue from its Github repository: https://github.com/googlecolab/colabtools/issues/299#issuecomment-615308778.

Answer 2

It's possible to run Docker in Colab, but with limiting functionality.

There are two methods of running Docker service, a regular one (more restrictive), and in rootless mode (dockerd inside RootlessKit).

dockerd

Install by:

!apt-get -qq install docker.io

Use the following shell script:

%%shell
set -x
dockerd -b none --iptables=0 -l warn &
for i in $(seq 5); do [ ! -S "/var/run/docker.sock" ] && sleep 2 || break; done
docker info
docker network ls
docker pull hello-world
docker pull ubuntu
# docker build -t myimage .
docker images
kill $(jobs -p)

As shown above, before each docker command, you've to run Docker service (dockerd) in the background, then kill it. Unfortunately you've to run dockerd for each cell where you want to run your docker commands.

Notes on dockerd arguments:

• -b none/--bridge none - Disables a network bridge to avoid errors.

• --iptables=0 - Disables addition of iptables rules to avoid errors.
• -D - Add to enable debug mode.

However in this mode running most of the containers will generate the errors related to read-only file system.

Additional notes:

• To disable cpuset support, run: !umount -vl /sys/fs/cgroup/cpuset.

Related issue: https://github.com/docker/for-linux/issues/1124.

Here are some notepads:

• https://colab.research.google.com/drive/1Lmbkc7v7XjSWK64E3NY1cw7iJ0sF1brl
• https://colab.research.google.com/drive/1RVS5EngPybRZ45PQRmz56PPdz9nWStIb (without cpuset support)

---

Rootless dockerd

Rootless mode allows running the Docker daemon and containers as a non-root user.

To install, use the following code:

%%shell
useradd -md /opt/docker docker
apt-get -qq install iproute2 uidmap
sudo -Hu docker SKIP_IPTABLES=1 bash < <(curl -fsSL https://get.docker.com/rootless)

To run dockerd service, there are two methods: using a script (dockerd-rootless.sh) or running rootlesskit directly.

Here is the script which uses dockerd-rootless.sh to run a hello-world container:

%%writefile docker-run.sh
#!/usr/bin/env bash
set -e
export DOCKER_SOCK=/opt/docker/.docker/run/docker.sock
export DOCKER_HOST=unix://$DOCKER_SOCK
export PATH=/opt/docker/bin:$PATH
export XDG_RUNTIME_DIR=/opt/docker/.docker/run
/opt/docker/bin/dockerd-rootless.sh --experimental --iptables=false --storage-driver vfs &
for i in $(seq 5); do [ ! -S "$DOCKER_SOCK" ] && sleep 2 || break; done
docker run $@
jobs -p
kill $(jobs -p)

To run above script, run:

!sudo -Hu docker bash -x docker-run.sh hello-world

The above may generate the following warnings:

WARN[0000] failed to mount sysfs, falling back to read-only mount: operation not permitted

To remount some folders with write access, you can try:

!mount -vt sysfs sysfs /sys -o rw,remount
!mount -vt tmpfs tmpfs /sys/fs/cgroup -o rw,remount

[rootlesskit:child ] error: executing [[ip tuntap add name tap0 mode tap] [ip link set tap0 address 02:50:00:00:00:01]]: exit status 1

The above error is related to dockerd-rootless.sh script which adds extra network parameters to rootlesskit such as:

--net=vpnkit --mtu=1500 --slirp4netns-sandbox=auto --slirp4netns-seccomp=auto --disable-host-loopback --port-driver=builtin

This has been reported at https://github.com/rootless-containers/rootlesskit/issues/181 (however ignored).

To workaround the above problem, we can pass our own arguments to rootlesskit using the following file instead:

%%writefile docker-run.sh
#!/usr/bin/env bash
set -e
export DOCKER_SOCK=/opt/docker/.docker/run/docker.sock
export DOCKER_HOST=unix://$DOCKER_SOCK
export PATH=/opt/docker/bin:$PATH
export XDG_RUNTIME_DIR=/opt/docker/.docker/run
rootlesskit --debug --disable-host-loopback --copy-up=/etc --copy-up=/run /opt/docker/bin/dockerd -b none --experimental --iptables=false --storage-driver vfs &
for i in $(seq 5); do [ ! -S "$DOCKER_SOCK" ] && sleep 2 || break; done
docker $@
jobs -p
kill $(jobs -p)

Then run as:

!sudo -Hu docker bash docker-run.sh run --cap-add SYS_ADMIN hello-world

Depending on your image, this may generate the following error:

process_linux.go:449: container init caused "join session keyring: create session key: operation not permitted": unknown.

Which could be solved by !sysctl -w kernel.keys.maxkeys=500, however Colab doesn't allow it. Related: Error response from daemon: join session keyring: create session key: disk quota exceeded.

Notepad showing the above:

• https://colab.research.google.com/drive/1oRja4v-PtY6lFMJIIF79No4s3s-vbqd4

Suggested further reading:

• Finding the minimal set of privileges for a docker container.