Execute Spring buildpacks when calling docker-compose build command

Question

I'm using Spring buildpacks introduced by Spring Boot 2.3.0.M1 for creating Docker images of Spring based applications. Everything goes well, I can create docker images for each application by executing ./gradlew bootBuildImage Gradle task, point docker-compose file to created images (e.g. image: spring-test:latest) and at the end successfully run all applications (docker-compose up).

Even though I have a bash script to automate the build process, I'd like to get rid of this additional step and make the Spring buildpacks task executed automatically whenever I run docker-compose up --build command thus the docker image of each application would get built and uploaded to the host's local docker repository from where it would be taken over by docker compose.

My first try was to create a dummy Dockerfile for each application which executes the bootBuildImage task on the host, but that would require an SSH connection from docker to host and not even sure that would work correctly.

Another thought was to use a similar approach, only change is to first mount or copy the app's source code to docker, configure buildpacks to store image into host's local docker image repo (perhaps SSH connection) and finally execute buildpacks on docker.

I'm wondering if there isn't a better, more elegant solution though.

Answer 1

This question really drove me nuts, since I've been playing around with Spring Boot & Paketo Buildpacks for quite a while now - and I really love the simplicity of Docker-Compose. So the questions was already inside the back of my head, but then you asked it :)

I didn't found a 100% perfect solution, but I think a have some ideas. Let's assume a example project of multiple Spring Boot apps, that are composed by a Maven multi-module setup (I know you're using Gradle, but there's a guide on doing multi module setups with Gradle over at the great spring.io guides): github.com/jonashackt/cxf-spring-cloud-netflix-docker. I created a new branch buildpacks-paketo containing all we need - and removed all Dockerfiles from the respective Spring Boot apps. Since we shouldn't need them anymore using Cloud Native Buildpacks (which is kind of their design goal).

TLDR: My idea is to use the spring-boot-maven-plugin (or it's Gradle equivalent) to issue a fresh Paketo build before every "normal docker-compose up like this:

mvn clean spring-boot:build-image && docker-compose up

The example projects parent pom.xml look like this (shortened):

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>de.jonashackt</groupId>
    <artifactId>cxf-spring-cloud-netflix-docker-build-all</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <packaging>pom</packaging>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.4.1</version>
    </parent>


    <modules>
        <module>eureka-serviceregistry</module>
        <module>spring-boot-admin</module>
        <module>zuul-edgeservice</module>
        <module>weatherbackend</module>
        <module>weatherservice</module>
        <module>weatherclient</module>
    </modules>

</project>

The example projects docker-compose.yml looks straightforward and uses the container images producted by Paketo (which is triggered by the Maven plugin) - those are named like this: eureka-serviceregistry:0.0.1-SNAPSHOT. Here's the full docker-compose.yml for all Spring Boot services:

version: '3.3'

services:

  eureka-serviceregistry:
    image: eureka-serviceregistry:0.0.1-SNAPSHOT
    ports:
     - "8761:8761"
    tty:
      true
    restart:
      unless-stopped

  spring-boot-admin:
    image: spring-boot-admin:0.0.1-SNAPSHOT
    ports:
     - "8092:8092"
    environment:
      - REGISTRY_HOST=eureka-serviceregistry
    tty:
      true
    restart:
      unless-stopped

  # no portbinding here - the actual services should be accessible through Zuul proxy
  weatherbackend:
    image: weatherbackend:0.0.1-SNAPSHOT
    ports:
     - "8090"
    environment:
      - REGISTRY_HOST=eureka-serviceregistry
    tty:
      true
    restart:
      unless-stopped

  # no portbinding here - the actual services should be accessible through Zuul proxy
  weatherservice:
    image: weatherservice:0.0.1-SNAPSHOT
    ports:
     - "8095"
    environment:
      - REGISTRY_HOST=eureka-serviceregistry
    tty:
      true
    restart:
      unless-stopped

  zuul-edgeservice:
    image: zuul-edgeservice:0.0.1-SNAPSHOT
    ports:
     - "8080:8080"
    environment:
      - REGISTRY_HOST=eureka-serviceregistry
    tty:
      true
    restart:
      unless-stopped

=== Possible enhancements =====================

The idea stuck me to also have "on single docker-compose.yml and only use docker-compose up as you asked for - no additional command. Therefore I created another "Docker Compose build service" that should only build the service images like this:

version: '3.3'

services:

  paketo-build:
    image: maven:3.6-openjdk-15
    command: "mvn clean spring-boot:build-image -B -DskipTests --no-transfer-progress" # build all apps
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro" # Mount Docker from host into build container for Paketo to work
      - "$HOME/.m2:/root/.m2" # Mount your local Maven repository into build container to prevent repeated downloads
      - "$PWD:/workspace" # Mount all Spring Boot apps into the build container
    working_dir: "/workspace"

I first integrated this service in the docker-compose.yml I already had. Running a docker-compose up paketo-build did what I was looking for: building all our Spring Boot apps inside the Compose setup:

...
paketo-build_1  | [INFO] --- spring-boot-maven-plugin:2.4.1:build-image (default-cli) @ eureka-serviceregistry ---
paketo-build_1  | [INFO] Building image 'docker.io/library/eureka-serviceregistry:0.0.1-SNAPSHOT'
paketo-build_1  | [INFO]
paketo-build_1  | [INFO]  > Pulling builder image 'docker.io/paketobuildpacks/builder:base' 100%
paketo-build_1  | [INFO]  > Pulled builder image 'paketobuildpacks/builder@sha256:3cff90d13d353ffdb83acb42540dae4ce6c97d55c07fb01c39fe0922177915fa'
paketo-build_1  | [INFO]  > Pulling run image 'docker.io/paketobuildpacks/run:base-cnb' 100%
paketo-build_1  | [INFO]  > Pulled run image 'paketobuildpacks/run@sha256:f393fa2927a2619a10fc09bb109f822d20df909c10fed4ce3c36fad313ea18e3'
paketo-build_1  | [INFO]  > Executing lifecycle version v0.10.1
paketo-build_1  | [INFO]  > Using build cache volume 'pack-cache-9d8694845b92.build'
paketo-build_1  | [INFO]
paketo-build_1  | [INFO]  > Running creator
paketo-build_1  | [INFO]     [creator]     ===> DETECTING
...
paketo-build_1  | [INFO]     [creator]
paketo-build_1  | [INFO]     [creator]     Paketo Spring Boot Buildpack 3.5.0
paketo-build_1  | [INFO]     [creator]       https://github.com/paketo-buildpacks/spring-boot
paketo-build_1  | [INFO]     [creator]       Creating slices from layers index
...
paketo-build_1  | [INFO]     [creator]     Adding label 'io.buildpacks.project.metadata'
paketo-build_1  | [INFO]     [creator]     Adding label 'org.opencontainers.image.title'
paketo-build_1  | [INFO]     [creator]     Adding label 'org.opencontainers.image.version'
paketo-build_1  | [INFO]     [creator]     Adding label 'org.springframework.boot.spring-configuration-metadata.json'
paketo-build_1  | [INFO]     [creator]     Adding label 'org.springframework.boot.version'
paketo-build_1  | [INFO]     [creator]     Setting default process type 'web'
paketo-build_1  | [INFO]     [creator]     *** Images (7efae8be1167):
paketo-build_1  | [INFO]     [creator]           docker.io/library/eureka-serviceregistry:0.0.1-SNAPSHOT
paketo-build_1  | [INFO]
paketo-build_1  | [INFO] Successfully built image 'docker.io/library/eureka-serviceregistry:0.0.1-SNAPSHOT'
...

But that didn't feel right because of a bunch of reasons. One is that you need to somehow wait with the startup of all other Compose services until the paketo-build service did it's job and build all the images. BUT as the Docker docs tell us, we will need to work against design decisions made in Compose to make that happen! Also did I find this great answer, where Max explains that it's not a good design to "pollute" the "production" docker-compose.yml with containers that are solely there for the build.

After that I extracted the paketo-build service into it's own Compose file - called build.yml inside the example project. With that we're now able to run the Paketo build without relying on the host to have Maven installed - and solely with Docker-Compose:

docker-compose -f build.yml up && docker-compose up

Remember to not detach from the first container with -d since the full Paketo build has to be finished before we start our docker-compose.yml. With this approach we also need absolutely no Dockerfile. But at the same time the idea came to me to remove the need for a separate build container fully and simply use Maven (or Gradle) before the up concatenated by a && like already described in the TLDR:

mvn clean spring-boot:build-image && docker-compose up

Hope this is of help to you. Would be glad to hear your feedback! Here's also a full GitHub actions build showing all the "magic" on a Cloud CI server.

Right now there's afaik no way to use docker-compose up --build to trigger a fresh image build of all your Spring Boot apps using Paketo Buildpacks.