deleting old images in Docker - OSX

Question

I've been toying with a docker image for Tensorflow.

To summarize, I first installed the standard image, then realized I needed nodejs, so added it and did a docker commit. Then realized I needed expressJS, added it an did a commit

I am running docker v1.12.5 (so the new gc/prune commands are not there)

At this stage, docker images -a shows:

REPOSITORY                     TAG                 IMAGE ID            CREATED             SIZE
tensor-node-express            latest              f2f59eb61aae        15 hours ago        2.104 GB
gcr.io/tensorflow/tensorflow   latest-devel        308238445d5c        2 days ago          1.995 GB
gcr.io/tensorflow/tensorflow   <none>              74435614a991        9 days ago          1.52 GB

I only want to keep tensor-node-express and delete the older images.

$ docker rmi 308238445d5c
Error response from daemon: conflict: unable to delete 308238445d5c (cannot be forced) - image has dependent child images

$docker rmi gcr.io/tensorflow/tensorflow:latest-devel
Error response from daemon: conflict: unable to remove repository reference "gcr.io/tensorflow/tensorflow:latest-devel" (must force) - container 03de9d864e31 is using its referenced image 308238445d5c

I assumed that this means docker commits store differential images, but when I go to ~/.docker/machine/machines/default, I see:

40894464 Mar 13 13:57 boot2docker.iso
5043847168 Mar 16 08:34 disk.vmdk

I suppose the 5G file is a composite of my images, which seems to show each docker commit is the full image!

Any thoughts on how I can only use the latest docker image (tensor-node-express) and free my HD of the invasion of docker?

Supplementary info - here is the output of docker ps -a

CONTAINER ID        IMAGE                                       COMMAND             CREATED             STATUS                        PORTS                NAMES
e6dcd2915991        tensor-node-express                         "/bin/bash"         15 hours ago        Exited (130) 15 hours ago                          flamboyant_bose
fb44b19a21c2        gcr.io/tensorflow/tensorflow:latest-devel   "/bin/bash"         18 hours ago        Exited (130) 15 hours ago                          compassionate_bose
075001a687e3        gcr.io/tensorflow/tensorflow:latest-devel   "/bin/bash"         18 hours ago        Exited (0) 18 hours ago                            nervous_sinoussi
a80ce2d2e688        gcr.io/tensorflow/tensorflow:latest-devel   "/bin/bash"         19 hours ago        Exited (130) 18 hours ago                          happy_euclid
f493bd3c8712        gcr.io/tensorflow/tensorflow:latest-devel   "/bin/bash"         19 hours ago        Exited (1) 19 hours ago                            friendly_cori
03de9d864e31        gcr.io/tensorflow/tensorflow:latest-devel   "/bin/bash"         2 days ago          Exited (255) 23 minutes ago   6006/tcp, 8888/tcp   tender_hopper
2dd1e83d62d3        gcr.io/tensorflow/tensorflow:latest-devel   "/bin/bash"         2 days ago          Exited (0) 15 hours ago                            modest_einstein
3067ed171b1c        gcr.io/tensorflow/tensorflow:latest-devel   "/bin/bash"         2 days ago          Exited (0) 2 days ago                              dazzling_bhabha
62c699afd3fd        74435614a991                                "/bin/bash"         2 days ago          Exited (127) 2 days ago                            inspiring_austin
9523ffe2945c        74435614a991                                "/bin/bash"         2 days ago          Exited (0) 2 days ago                              kickass_leakey
e06958ea517c        74435614a991                                "/bin/bash"         2 days ago          Exited (0) 2 days ago                              objective_euler
ccf922954667        74435614a991                                "/bin/bash"         2 days ago          Exited (255) 2 days ago                            dreamy_bartik
fad0d92a07a3        74435614a991                                "/bin/bash"         2 days ago          Exited (130) 2 days ago                            elastic_dubinsky
f2a98d4e11ea        74435614a991                                "/bin/bash"         2 days ago          Exited (0) 2 days ago                              heuristic_kilby
f07e46367b17        74435614a991                                "/bin/bash"         2 days ago          Exited (130) 2 days ago                            trusting_darwin
5bbf9cf992b8        74435614a991                                "/bin/bash"         2 days ago          Exited (0) 2 days ago                              flamboyant_knuth

I tried docker ps --filter "status=exited" | grep "days ago" | awk '{print $1}' | xargs docker rm (credit)

I ran the above manually as well for some of the containers it missed

That pruned the ps list to:

docker ps -a
CONTAINER ID        IMAGE                 COMMAND             CREATED             STATUS                      PORTS               NAMES
e6dcd2915991        tensor-node-express   "/bin/bash"         15 hours ago        Exited (130) 15 hours ago                       flamboyant_bose

But even then I can't delete old images - same error.

Further update, I tried to list dependencies in images (credit) using this script:

  for i in $(docker images -q)
    do
        docker history $i | grep -q 74435614a991 && echo $i
    done | sort -u

And it told me:

308238445d5c
74435614a991
f2f59eb61aae

This means my new images are child images of the old image. But the size is not a differential looking at the disk size.

Thoughts?

Answer 1

docker-machine uses a Linux VM

When you looked at the docker-machine .vdmk and .iso files, what you are looking at is files for a Linux VM running on your Mac. This is needed because Docker requires Linux kernel features to run, it cannot run directly on the Mac's microkernel.

So your Mac is running a Linux virtual machine, and inside that virutal machine is running the Docker daemon and all of your containers.

Therefore the file size of the .vmdk and .iso tell you nothing about any one image.

docker images have parent/child relationships

As you may already know, docker images have parents and/or children. For instance when you build an image with a Dockerfile like this:

FROM ubuntu:latest
RUN apt-get update && apt-get install nginx

You will end up with a new image that you have perhaps tagged my-nginx. But it requires the ubuntu:latest image as its parent; you cannot delete ubuntu:latest with this image still around, as it requires its parent.

docker commit creates those relationships

When you use docker commit, you are basically doing a dynamic snapshot build. It is similar to the above, except there's no Dockerfile involved.

The above example has a FROM line which indicates the image to use as a base. When using commit, there is a base implied - whatever image was used to launch the running container that you are committing.

The above example has a RUN command which will create new contents in the built image, above and beyond the base image. In a real Dockerfile there are usually multiple commands that do various things which build on the base image. When you use commit, you don't have that. Instead, anything that has been written to the container on top of the base image is your new content. It exists in a read-write filesystem layer in the container. That is the thing you are committing; it is written as a new read-only layer and you get that back as a new (immutable, read-only) docker image. With a parent.

Based on your comments, and the question itself, you appear to have believed that using docker commit would create a new full image that had no dependencies on other images. That is not true. You can craft images like that if you build them yourself from scratch, just not this way.

You can untag the image

If what you want is for the image to not show up in your list, that's easy. Just untag it.

docker rmi gcr.io/tensorflow/tensorflow:latest-devel

However, this is more or less cosmetic. The image will still be there, as another image requires it. All this does is remove the tag, so it doesn't appear in the docker images list anymore without the -a flag.

The reason trying this did not work for you is you tried to rmi the image using its ID, not using its tag.