How do I get Java testcontainers to work in Docker Multistage builds?

Question

I have a problem similar to Run (Docker) Test Container in gitlab with Maven. The difference is that rather than my script running mvn directly it runs a docker multistage build that runs the test inside of the docker image. Unfortunately this doesn't appear to work for the PostgreSQL Test Container.

Dockerfile

#############
### build ###
#############

# base image
FROM maven:3-jdk-11 as build

# set working directory
WORKDIR /app

# add app
COPY . .

RUN export MAVEN_OPTS="-Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=WARN -Dorg.slf4j.simpleLogger.showDateTime=true -Djava.awt.headless=true" && export MAVEN_CLI_OPTS="-B -U --batch-mode --errors --fail-at-end --show-version -DinstallAtEnd=true -DdeployAtEnd=true"

RUN mvn $MAVEN_CLI_OPTS clean install

############
### prod ###
############

# Yea this isn't right, but it crashes before it gets to this point. This is for example purposes only.
FROM openjdk:15-jdk-alpine
COPY --from=build /app/reproducer-testcontainer/target/reproducer-testcontainer.jar /reproducer-testcontainer.jar
CMD java -jar reproducer-testcontainer.jar

When I run mvn clean install it works properly and runs my test using the PostgreSQL Test Container. However, when I run docker build . it fails at the mvn clean install step with the below stack trace.

Stack trace:

13:05:01.250 [main] ERROR org.testcontainers.dockerclient.EnvironmentAndSystemPropertyClientProviderStrategy - ping failed with configuration Environment variables, system properties and defaults. Resolved: 
     dockerHost=unix:///var/run/docker.sock
     apiVersion='{UNKNOWN_VERSION}'
     registryUrl='https://index.docker.io/v1/'
     registryUsername='root'
     registryPassword='null'
     registryEmail='null'
     dockerConfig='DefaultDockerClientConfig[dockerHost=unix:///var/run/docker.sock,registryUsername=root,registryPassword=<null>,registryEmail=<null>,registryUrl=https://index.docker.io/v1/,dockerConfigPath=/root/.docker,sslConfig=<null>,apiVersion={UNKNOWN_VERSION},dockerConfig=<null>]'
  due to org.rnorth.ducttape.TimeoutException: Timeout waiting for result with exception
 org.rnorth.ducttape.TimeoutException: Timeout waiting for result with exception
    at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:51)
<snip>
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
 Caused by: java.io.IOException: com.sun.jna.LastErrorException: [2] No such file or directory
    at org.testcontainers.shaded.org.scalasbt.ipcsocket.UnixDomainSocket.<init>(UnixDomainSocket.java:62)
<snip>
    at java.base/java.lang.Thread.run(Thread.java:834)
 Caused by: com.sun.jna.LastErrorException: [2] No such file or directory
    at org.testcontainers.shaded.org.scalasbt.ipcsocket.UnixDomainSocketLibrary.connect(Native Method)
    at org.testcontainers.shaded.org.scalasbt.ipcsocket.UnixDomainSocket.<init>(UnixDomainSocket.java:57)
    ... 35 common frames omitted

In my CI pipeline I'd like to only run docker build . and not worry about having another stage that does the mvn clean install.

How do I fix the configuration to get the java PostgreSQL Testcontainers to work inside of a Docker build so that I can use it in a multi-stage build?

Full Code example: https://gitlab.com/raymondcg/reproducer-testcontainer

Answer 1

This is specifically for Docker Desktop for Windows. First you need to expose the Docker daemon

docker run -p 2375:2375 -v //var/run/docker.sock://var/run/docker.sock alpine/socat tcp-listen:2375,reuseaddr,fork unix-connect:/var/run/docker.sock

# For extra security

docker run -p 127.0.0.1:2375:2375 -v //var/run/docker.sock://var/run/docker.sock alpine/socat tcp-listen:2375,reuseaddr,fork unix-connect:/var/run/docker.sock

For Docker Desktop for macOS, same process as Windows. First you need to expose the Docker daemon (just no // since there's no need to escape anymore)

docker run -p 2375:2375 -v /var/run/docker.sock:/var/run/docker.sock alpine/socat tcp-listen:2375,reuseaddr,fork unix-connect:/var/run/docker.sock

Then in your Dockerfile you have

ENV DOCKER_HOST=tcp://host.docker.internal:2375 
ENV TESTCONTAINERS_HOST_OVERRIDE=host.docker.internal 
RUN gradle --no-daemon test

For Linux which does not have host.docker.internal it can be emulated by adding the host mapping

docker build . --add-host host.docker.internal:host-gateway

Answer 2

Not really Testcontainers related.

Testcontainers requires a valid Docker daemon. When you build images, there is no daemon mounted into the image build context.

You can easily verify that by doing:

RUN curl --unix-socket /var/run/docker.sock http:/_/_ping

Make this command return "OK" (no need to run the Testcontainers code), and your tests will pass as well.