How to ignore folders to send in docker build context

Question

I am facing an issue of large docker build context because of my project structure. In my root directory I have lib folder for common code and folders of micro-services. Now I want to build for miscroservice1 to include only lib folder and to ignore other microservices.

I am running docker build command in root folder because running command in microservice folder giving error Forbidden path outside the build context

rootFolder
-- lib
-- microservice1/Dockerfile
-- microservice2/Dockerfile
-- microservice3/Dockerfile

I have two solutions but didn't try for now

1. To add symlinks for lib in my microservice folder
2. To write script for each docker build to add lib folder in specific microservice folder and then run docker build.

I am trying the above two solutions. Can anyone suggest any best practice?

Answer 1

You can create .dockerignore in your root directory and add

microservice1/
microservice2/
microservice3/

to it, just like .gitignore does during tracking files, docker will ignore these folders/files during the build.

Update

You can include docker-compose.yml file in your root directory, look at docker-compose for all the options, such as setting environment, running a specific command, etc, that you can use during the build process.

version: "3"
services:
  microservice1:
    build:
      context: .
      dockerfile: ./microservice1/Dockerfile
    volumes:
      - "./path/to/share:/path/to/mount/on/container"
    ports:
      - "<host>:<container>"
    links:
      - rootservice # defines a dns record in /etc/hosts to point to rootservice
  microservice2:
    build:
      context: .
      dockerfile: ./microservice2/Dockerfile
    volumes:
      - "./path/to/share:/path/to/mount/on/container"
    ports:
      - "<host>:<container>"
    links:
      - rootservice # defines a dns record in /etc/hosts to point to rootservice
      - microservice1
  rootservice:
    build:
      context: .
      dockerfile: ./Dockerfile
    volumes:
      - "./path/to/share:/path/to/mount/on/container"
    ports:
      - "<host>:<container>"
    depends_on:
      - microservice1
      - microservice2
    ports:
      - "<host1>:<container1>"
      - "<host2>:<container2>"

This will be your build recipe for your microservices, you can now run docker-compose build to build all your images.

Answer 2

If the only tool you have is Docker, there aren't very many choices. The key problem is that there is only one .dockerignore file. That means you always have to use your project root directory as the Docker context directory (including every services' sources), but you can tell Docker which specific Dockerfile within that to use. (Note that all COPY directives will be relative to the rootFolder in this case.)

docker build rootFolder -f microservice1/Dockerfile -t micro/service1:20190831.01

In many languages there is a way to package up the library (C .a, .h, and .so files; Java .jar files; Python wheels; ...). If your language supports that, another option is to build the library, then copy (not symlink) the library into each service's build tree. Using Python's wheel format as an example:

pip wheel ./lib
cp microlib.whl microservice1
docker build microservice1 -t micro/service1:20190831.01
# Dockerfile needs to
# RUN pip install ./microlib.whl

Another useful variant on this is a manual multi-stage build. You can have lib/Dockerfile pick some base image, and then install the library into that base image. Then each service's Dockerfile starts FROM the library image, and has it preinstalled. Using a C library as an example:

# I am lib/Dockerfile

# Build stage
FROM ubuntu:18.04 AS build
RUN apt-get update && apt-get install build-essential
WORKDIR /src
COPY ./ ./
RUN ./configure --prefix=/usr/local && make
# This is a typical pattern implemented by GNU Autoconf:
# it actually writes files into /src/out/usr/local/...
RUN make install DESTDIR=/src/out

# Install stage -- service images are based on this
FROM ubuntu:18.04
COPY --from=build /src/out /
RUN ldconfig

# I am microservice1/Dockerfile
ARG VERSION=latest
FROM micro/lib:${VERSION}
# From the base image, there are already
# /usr/local/include/microlib.h and /usr/local/lib/libmicro.so
COPY ...
RUN gcc ... -lmicro
CMD ...

There is also usually an option (again, depending on your language and its packaging system) to upload your built library to some server, possibly one you're running yourself. (A Python pip requirements.txt file can contain an arbitrary HTTP URL for a wheel, for example.) If you do this then you can just declare your library as an ordinary dependency, and this problem goes away.

Which of these works better for you depends on your language and runtime, and how much automation of multiple coordinated docker build commands you're willing to do.