Running Docker on Google Colab

Question

I'm trying to run Docker on Google Colab, and I'm having some trouble. I followed the steps to install Docker on Ubuntu (https://docs.docker.com/engine/install/ubuntu/), which are very similar to the steps that others have used for installing it on Digital Ocean (https://colab.research.google.com/drive/10OinT5ZNGtdLLQ9K399jlKgNgidxUbGP#scrollTo=9zp-VmLsws2r).

I'm able to successfully install Docker—I know this because when I run docker --version I get the output: Docker version 19.03.11, build 42e35e61f3, and I never get any error saying that the command docker isn't recognized.

But when I try to run sudo docker run hello-world --priveleged, I get the output: docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?..

I did some diagnostics by running sudo dockerd --debug, and the output is below. I have checked, and both iptables and my kernel are fully upgraded. Running whoami returns root. Some have recommended restarting the kernel, which also didn't help.

This seems like a Google Colab-specific issue, but I can't figure out exactly what's wrong. Does anyone have any thoughts? Please let me know if you need any more info.

Thanks,

Rob

Here is exactly the code I ran in Google Colab:

%%shell
sudo curl -fsSL https://get.docker.com/ | sh
sudo usermod -aG docker root
sudo docker run hello-world --priveleged
sudo dockerd --debug

And here is the error:

INFO[2020-06-06T22:08:10.430389079Z] Starting up                                  
DEBU[2020-06-06T22:08:10.430968786Z] Listener created for HTTP on unix (/var/run/docker.sock) 
DEBU[2020-06-06T22:08:10.430993252Z] Containerd not running, starting daemon managed containerd 
INFO[2020-06-06T22:08:10.431768021Z] libcontainerd: started new containerd process  pid=19853
INFO[2020-06-06T22:08:10.431817137Z] parsed scheme: "unix"                         module=grpc
INFO[2020-06-06T22:08:10.431844607Z] scheme "unix" not registered, fallback to default scheme  module=grpc
INFO[2020-06-06T22:08:10.431872314Z] ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0  <nil>}] <nil>}  module=grpc
INFO[2020-06-06T22:08:10.431886205Z] ClientConn switching balancer to "pick_first"  module=grpc
INFO[2020-06-06T22:08:10.460579142Z] starting containerd                           revision=7ad184331fa3e55e52b890ea95e65ba581ae3429 version=1.2.13
DEBU[2020-06-06T22:08:10.460646512Z] changing OOM score to -500                   
ERRO[2020-06-06T22:08:10.460696793Z] failed to change OOM score to -500            error="write /proc/19853/oom_score_adj: permission denied"
INFO[2020-06-06T22:08:10.460996041Z] loading plugin "io.containerd.content.v1.content"...  type=io.containerd.content.v1
INFO[2020-06-06T22:08:10.461036833Z] loading plugin "io.containerd.snapshotter.v1.btrfs"...  type=io.containerd.snapshotter.v1
WARN[2020-06-06T22:08:10.461276128Z] failed to load plugin io.containerd.snapshotter.v1.btrfs  error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"
INFO[2020-06-06T22:08:10.461294855Z] loading plugin "io.containerd.snapshotter.v1.aufs"...  type=io.containerd.snapshotter.v1
WARN[2020-06-06T22:08:10.462332024Z] failed to load plugin io.containerd.snapshotter.v1.aufs  error="modprobe aufs failed: "modprobe: FATAL: Module aufs not found in directory /lib/modules/4.19.104+\n": exit status 1"
INFO[2020-06-06T22:08:10.462354904Z] loading plugin "io.containerd.snapshotter.v1.native"...  type=io.containerd.snapshotter.v1
INFO[2020-06-06T22:08:10.462404944Z] loading plugin "io.containerd.snapshotter.v1.overlayfs"...  type=io.containerd.snapshotter.v1
INFO[2020-06-06T22:08:10.462528569Z] loading plugin "io.containerd.snapshotter.v1.zfs"...  type=io.containerd.snapshotter.v1
INFO[2020-06-06T22:08:10.462929351Z] skip loading plugin "io.containerd.snapshotter.v1.zfs"...  type=io.containerd.snapshotter.v1
INFO[2020-06-06T22:08:10.462950017Z] loading plugin "io.containerd.metadata.v1.bolt"...  type=io.containerd.metadata.v1
WARN[2020-06-06T22:08:10.462975101Z] could not use snapshotter btrfs in metadata plugin  error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"
WARN[2020-06-06T22:08:10.462987603Z] could not use snapshotter aufs in metadata plugin  error="modprobe aufs failed: "modprobe: FATAL: Module aufs not found in directory /lib/modules/4.19.104+\n": exit status 1"
WARN[2020-06-06T22:08:10.463001160Z] could not use snapshotter zfs in metadata plugin  error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter: skip plugin"
INFO[2020-06-06T22:08:10.463088275Z] loading plugin "io.containerd.differ.v1.walking"...  type=io.containerd.differ.v1
INFO[2020-06-06T22:08:10.463110086Z] loading plugin "io.containerd.gc.v1.scheduler"...  type=io.containerd.gc.v1
INFO[2020-06-06T22:08:10.463144139Z] loading plugin "io.containerd.service.v1.containers-service"...  type=io.containerd.service.v1
INFO[2020-06-06T22:08:10.463167701Z] loading plugin "io.containerd.service.v1.content-service"...  type=io.containerd.service.v1
INFO[2020-06-06T22:08:10.463186973Z] loading plugin "io.containerd.service.v1.diff-service"...  type=io.containerd.service.v1
INFO[2020-06-06T22:08:10.463206483Z] loading plugin "io.containerd.service.v1.images-service"...  type=io.containerd.service.v1
INFO[2020-06-06T22:08:10.463227897Z] loading plugin "io.containerd.service.v1.leases-service"...  type=io.containerd.service.v1
INFO[2020-06-06T22:08:10.463246966Z] loading plugin "io.containerd.service.v1.namespaces-service"...  type=io.containerd.service.v1
INFO[2020-06-06T22:08:10.463264776Z] loading plugin "io.containerd.service.v1.snapshots-service"...  type=io.containerd.service.v1
INFO[2020-06-06T22:08:10.463283179Z] loading plugin "io.containerd.runtime.v1.linux"...  type=io.containerd.runtime.v1
INFO[2020-06-06T22:08:10.463370355Z] loading plugin "io.containerd.runtime.v2.task"...  type=io.containerd.runtime.v2
INFO[2020-06-06T22:08:10.463431282Z] loading plugin "io.containerd.monitor.v1.cgroups"...  type=io.containerd.monitor.v1
INFO[2020-06-06T22:08:10.463905587Z] loading plugin "io.containerd.service.v1.tasks-service"...  type=io.containerd.service.v1
INFO[2020-06-06T22:08:10.463938829Z] loading plugin "io.containerd.internal.v1.restart"...  type=io.containerd.internal.v1
INFO[2020-06-06T22:08:10.463987516Z] loading plugin "io.containerd.grpc.v1.containers"...  type=io.containerd.grpc.v1
INFO[2020-06-06T22:08:10.464006257Z] loading plugin "io.containerd.grpc.v1.content"...  type=io.containerd.grpc.v1
INFO[2020-06-06T22:08:10.464023427Z] loading plugin "io.containerd.grpc.v1.diff"...  type=io.containerd.grpc.v1
INFO[2020-06-06T22:08:10.464039729Z] loading plugin "io.containerd.grpc.v1.events"...  type=io.containerd.grpc.v1
INFO[2020-06-06T22:08:10.464056143Z] loading plugin "io.containerd.grpc.v1.healthcheck"...  type=io.containerd.grpc.v1
INFO[2020-06-06T22:08:10.464076162Z] loading plugin "io.containerd.grpc.v1.images"...  type=io.containerd.grpc.v1
INFO[2020-06-06T22:08:10.464092665Z] loading plugin "io.containerd.grpc.v1.leases"...  type=io.containerd.grpc.v1
INFO[2020-06-06T22:08:10.464138115Z] loading plugin "io.containerd.grpc.v1.namespaces"...  type=io.containerd.grpc.v1
INFO[2020-06-06T22:08:10.464157926Z] loading plugin "io.containerd.internal.v1.opt"...  type=io.containerd.internal.v1
INFO[2020-06-06T22:08:10.464214011Z] loading plugin "io.containerd.grpc.v1.snapshots"...  type=io.containerd.grpc.v1
INFO[2020-06-06T22:08:10.464232479Z] loading plugin "io.containerd.grpc.v1.tasks"...  type=io.containerd.grpc.v1
INFO[2020-06-06T22:08:10.464250571Z] loading plugin "io.containerd.grpc.v1.version"...  type=io.containerd.grpc.v1
INFO[2020-06-06T22:08:10.464267665Z] loading plugin "io.containerd.grpc.v1.introspection"...  type=io.containerd.grpc.v1
INFO[2020-06-06T22:08:10.464535499Z] serving...                                    address="/var/run/docker/containerd/containerd-debug.sock"
INFO[2020-06-06T22:08:10.464610161Z] serving...                                    address="/var/run/docker/containerd/containerd.sock"
INFO[2020-06-06T22:08:10.464628619Z] containerd successfully booted in 0.004750s  
DEBU[2020-06-06T22:08:10.478839838Z] Started daemon managed containerd            
DEBU[2020-06-06T22:08:10.479163732Z] Permission denied writing "-500" to /proc/self/oom_score_adj 
DEBU[2020-06-06T22:08:10.479491477Z] Golang's threads limit set to 93420          
INFO[2020-06-06T22:08:10.479914598Z] parsed scheme: "unix"                         module=grpc
INFO[2020-06-06T22:08:10.479950021Z] scheme "unix" not registered, fallback to default scheme  module=grpc
INFO[2020-06-06T22:08:10.479994253Z] ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0  <nil>}] <nil>}  module=grpc
INFO[2020-06-06T22:08:10.480008702Z] ClientConn switching balancer to "pick_first"  module=grpc
INFO[2020-06-06T22:08:10.481507826Z] parsed scheme: "unix"                         module=grpc
INFO[2020-06-06T22:08:10.481551933Z] scheme "unix" not registered, fallback to default scheme  module=grpc
INFO[2020-06-06T22:08:10.481574834Z] ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0  <nil>}] <nil>}  module=grpc
INFO[2020-06-06T22:08:10.481587896Z] ClientConn switching balancer to "pick_first"  module=grpc
DEBU[2020-06-06T22:08:10.482587875Z] Using default logging driver json-file       
DEBU[2020-06-06T22:08:10.482634103Z] [graphdriver] priority list: [btrfs zfs overlay2 aufs overlay devicemapper vfs] 
DEBU[2020-06-06T22:08:10.482693139Z] zfs command is not available: exec: "zfs": executable file not found in $PATH  storage-driver=zfs
DEBU[2020-06-06T22:08:10.482712159Z] processing event stream                       module=libcontainerd namespace=plugins.moby
ERRO[2020-06-06T22:08:10.494846158Z] failed to mount overlay: invalid argument     storage-driver=overlay2
ERRO[2020-06-06T22:08:10.495940320Z] AUFS was not found in /proc/filesystems       storage-driver=aufs
ERRO[2020-06-06T22:08:10.508077799Z] failed to mount overlay: invalid argument     storage-driver=overlay
DEBU[2020-06-06T22:08:10.509386612Z] Initialized graph driver vfs                 
DEBU[2020-06-06T22:08:10.521417061Z] Max Concurrent Downloads: 3                  
DEBU[2020-06-06T22:08:10.521441888Z] Max Concurrent Uploads: 5                    
INFO[2020-06-06T22:08:10.521484213Z] Loading containers: start.                   
DEBU[2020-06-06T22:08:10.521565238Z] Option Experimental: false                   
DEBU[2020-06-06T22:08:10.521595561Z] Option DefaultDriver: bridge                 
DEBU[2020-06-06T22:08:10.521606624Z] Option DefaultNetwork: bridge                
DEBU[2020-06-06T22:08:10.521617533Z] Network Control Plane MTU: 1500              
DEBU[2020-06-06T22:08:10.521732715Z] processing event stream                       module=libcontainerd namespace=moby
WARN[2020-06-06T22:08:10.522980796Z] Running iptables --wait -t nat -L -n failed with message: `iptables v1.6.1: can't initialize iptables table `nat': Permission denied (you must be root)
Perhaps iptables or your kernel needs to be upgraded.`, error: exit status 3 
DEBU[2020-06-06T22:08:10.523046607Z] Fail to initialize firewalld: Failed to connect to D-Bus system bus: dial unix /var/run/dbus/system_bus_socket: connect: no such file or directory, using raw iptables instead 
DEBU[2020-06-06T22:08:10.525297489Z] /sbin/iptables, [-t filter -C FORWARD -j DOCKER-ISOLATION] 
DEBU[2020-06-06T22:08:10.526293151Z] /sbin/iptables, [-t nat -D PREROUTING -m addrtype --dst-type LOCAL -j DOCKER] 
DEBU[2020-06-06T22:08:10.527312194Z] /sbin/iptables, [-t nat -D OUTPUT -m addrtype --dst-type LOCAL ! --dst 127.0.0.0/8 -j DOCKER] 
DEBU[2020-06-06T22:08:10.528249067Z] /sbin/iptables, [-t nat -D OUTPUT -m addrtype --dst-type LOCAL -j DOCKER] 
DEBU[2020-06-06T22:08:10.529256673Z] /sbin/iptables, [-t nat -D PREROUTING]       
DEBU[2020-06-06T22:08:10.530097230Z] /sbin/iptables, [-t nat -D OUTPUT]           
DEBU[2020-06-06T22:08:10.530898377Z] /sbin/iptables, [-t nat -F DOCKER]           
DEBU[2020-06-06T22:08:10.531707985Z] /sbin/iptables, [-t nat -X DOCKER]           
DEBU[2020-06-06T22:08:10.532515381Z] /sbin/iptables, [-t filter -F DOCKER]        
DEBU[2020-06-06T22:08:10.533289600Z] /sbin/iptables, [-t filter -X DOCKER]        
DEBU[2020-06-06T22:08:10.534161170Z] /sbin/iptables, [-t filter -F DOCKER-ISOLATION-STAGE-1] 
DEBU[2020-06-06T22:08:10.534943495Z] /sbin/iptables, [-t filter -X DOCKER-ISOLATION-STAGE-1] 
DEBU[2020-06-06T22:08:10.535746652Z] /sbin/iptables, [-t filter -F DOCKER-ISOLATION-STAGE-2] 
DEBU[2020-06-06T22:08:10.536501715Z] /sbin/iptables, [-t filter -X DOCKER-ISOLATION-STAGE-2] 
DEBU[2020-06-06T22:08:10.537334216Z] /sbin/iptables, [-t filter -F DOCKER-ISOLATION] 
DEBU[2020-06-06T22:08:10.538112885Z] /sbin/iptables, [-t filter -X DOCKER-ISOLATION] 
DEBU[2020-06-06T22:08:10.538925285Z] /sbin/iptables, [-t nat -n -L DOCKER]        
DEBU[2020-06-06T22:08:10.539675733Z] /sbin/iptables, [-t nat -N DOCKER]           
DEBU[2020-06-06T22:08:10.540408597Z] daemon configured with a 15 seconds minimum shutdown timeout 
DEBU[2020-06-06T22:08:10.540428145Z] start clean shutdown of all containers with a 15 seconds timeout... 
DEBU[2020-06-06T22:08:10.540809633Z] Cleaning up old mountid : start.             
INFO[2020-06-06T22:08:10.540855630Z] stopping event stream following graceful shutdown  error="<nil>" module=libcontainerd namespace=moby
DEBU[2020-06-06T22:08:10.541282726Z] Cleaning up old mountid : done.              
INFO[2020-06-06T22:08:10.541490677Z] stopping healthcheck following graceful shutdown  module=libcontainerd
INFO[2020-06-06T22:08:10.541509355Z] stopping event stream following graceful shutdown  error="context canceled" module=libcontainerd namespace=plugins.moby
DEBU[2020-06-06T22:08:10.541598181Z] received signal                               signal=terminated
failed to start daemon: Error initializing network controller: error obtaining controller instance: failed to create NAT chain DOCKER: iptables failed: iptables -t nat -N DOCKER: iptables v1.6.1: can't initialize iptables table `nat': Permission denied (you must be root)
Perhaps iptables or your kernel needs to be upgraded.
 (exit status 3)

Answer 1

I was disappointed to find that Docker is not supported on Google Colab, and Google has no plans to support it.

https://github.com/googlecolab/colabtools/issues/299