Build and use docker image from docker-compose

Question

I have a single Dockerfile which creates an image with all of the dependencies that my images need to run. Then, I'm mounting files as read only data volumes, and setting various files as entrypoints. While this means that I don't need to have multiple Dockerfiles, for some reason every time I add a new service to the docker-compose file, I get a new image created (though it looks like it's just being pulled from cache). Is there a way to have the docker-compose file build the image once, and then have all of the services depend on that newly built image?

I know that it's possible to build the image separately, give it a name, and then have the docker-compose file reference that image, but that requires 2 commands rather than a single docker-compose up that I use right now.

Here is what my docker images output looks like. You can see that each service is getting its own image created, which I don't like. I'm not entirely sure whether this means that each image is taking up an additional 1.3gb on the disk, or if they're all just referencing the same base image (I'd guess the latter, but would love to know for sure), but it still pollutes the docker images list.

REPOSITORY             TAG                 IMAGE ID            CREATED             SIZE
docker_http_server_1   latest              a6eac9198c44        5 weeks ago         1.315 GB
docker_test_client     latest              a6eac9198c44        5 weeks ago         1.315 GB
docker_test_client_3   latest              a6eac9198c44        5 weeks ago         1.315 GB
meh                    latest              a6eac9198c44        5 weeks ago         1.315 GB
docker_data_server_1   latest              a6eac9198c44        5 weeks ago         1.315 GB
docker_server1         latest              a6eac9198c44        5 weeks ago         1.315 GB
docker_server2         latest              a6eac9198c44        5 weeks ago         1.315 GB
docker_server          latest              a6eac9198c44        5 weeks ago         1.315 GB
docker_test_client_2   latest              a6eac9198c44        5 weeks ago         1.315 GB
docker_http_server_2   latest              a6eac9198c44        5 weeks ago         1.315 GB
<none>                 <none>              9428e39bd080        5 weeks ago         431.7 MB
<none>                 <none>              08a27b512ded        5 weeks ago         430.3 MB
<none>                 <none>              acc3e230ecaa        5 weeks ago         411.1 MB
<none>                 <none>              96c74b6e7d9d        5 weeks ago         829.2 MB
<none>                 <none>              acf4a5ef1eeb        5 weeks ago         677 MB
<none>                 <none>              8f646f9a5352        5 weeks ago         2.759 GB
<none>                 <none>              ce8fa0a27cde        5 weeks ago         562.3 MB
<none>                 <none>              533cfe78e0d2        5 weeks ago         165.6 MB
redis                  latest              0d1cbfaa41da        7 weeks ago         185 MB
ubuntu                 16.04               bd3d4369aebc        7 weeks ago         126.6 MB
nginx                  latest              4efb2fcdb1ab        8 weeks ago         183.4 MB
hello-world            latest              c54a2cc56cbb        3 months ago        1.848 kB

Current docker-compose file:

version: '2'
services:
    data_server_1:
        build: .
        volumes:
            - "..:/buggy:ro"
        entrypoint: "/buggy/buggy_data_server.py"

    http_server_1:
        build: .
        volumes:
            - "..:/buggy:ro"
        entrypoint: "/buggy/buggy_http_server.py"
        links:
            - data_server_1 

    http_server_2:
        build: .
        volumes:
            - "..:/buggy:ro"
        entrypoint: "/buggy/buggy_http_server.py"
        links:
            - data_server_1 

    nginx:
        image: nginx
        ports:
            - "80:80"
            - "443:443"
            - "4242:4242"
        links:
            - data_server_1 
            - http_server_1 
            - http_server_2
        volumes:
            - "../nginx/nginx.conf:/etc/nginx/nginx.conf:ro"
            - "../static:/www/static"
            - "../protos:/www/protos"

    # This is effectively the same as running it on the host machine and host
    # network rather than within the docker network. This is so that we can test
    # to ensure that the tcp listening port works via the public interface
    # rather than through the docker channels.
    test_client:
        build: .
        network_mode: "host"
        volumes:
            - "..:/buggy:ro"
        entrypoint:
            - "/buggy/test_client.py"
            - "--key=/buggy/keys/robot1.key"
            - "--no-gui"
            - "--status"
            - "--buggy-name=Deep Mind (Status)"

    test_client_2:
        build: .
        network_mode: "host"
        volumes:
            - "..:/buggy:ro"
        entrypoint:
            - "/buggy/test_client.py"
            - "--key=/buggy/keys/robot2.key"
            - "--no-gui"
            - "--imu"
            - "--buggy-name=Bender (IMU)"

    test_client_3:
        build: .
        network_mode: "host"
        volumes:
            - "..:/buggy:ro"
        entrypoint:
            - "/buggy/test_client.py"
            - "--key=/buggy/keys/robot3.key"
            - "--no-gui"
            - "--camera"
            - "--buggy-name=Megatron (Camera)"

Also, if there are any suggestions on how to otherwise improve the docker-compose file, I'd love to hear them, but I believe that's a question for another time.

Answer 1

If you're looking for more sophisticated build options, you might want to look outside of Compose. As it's been said in other answers there are two concerns here (1. building images, 2. running containers). Compose is focused on running containers.

You could use dobi (disclaimer: I am the author of dobi) to build your images, and launch Compose. Your dobi.yaml might look like this:

image=app:
  image: reponame/appname
  tags: [latest]

compose=dev:
  files: [docker-compose.yml]
  project: '{project}'
  depends: [app]

Which you could run with dobi dev to build the images (once) and start Compose (the equivalent of docker-compose up -d. To get interactive logs you can run dobi dev:attach.

You'd also have to remove the build: . from your docker-compose.yml and replace it with image: reponame/appname:latest

Answer 2

I'm not entirely sure whether this means that each image is taking up an additional 1.3gb on the disk, or if they're all just referencing the same base image

They're all referencing the same base image as reflected by identical image IDs. Container names behave like symbolic links.

Is there a way to have the docker-compose file build the image once, and then have all of the services depend on that newly built image?

You could work around this issue by just building the image once and referencing it accordingly in your compose file, e.g.

version: '2'
services:
    data_server_1:
        build: .
        [...]

    http_server_1:
        image: docker_data_server_1
        [...]

    http_server_2:
        image: docker_data_server_1
        [...]

Be aware that is approach is very fragile. Renaming either the parent folder or the service itself will cause issues. If docker-compose spins up the services in the wrong order, starting some services will either fail or rely on a potentially outdated image.

I know that it's possible to build the image separately, give it a name, and then have the docker-compose file reference that image, but that requires 2 commands rather than a single docker-compose up that I use right now.

I'd still encourage you to do this. It makes things much more consistent (and faster!). Developing an image and testing a multi-container application are two different things and should be treated accordingly.