Why does the docker-compose healthcheck of my mongo container always fail?

Question

I'm using docker-compose to stand up an Express/React/Mongo app. I can currently stand up everything using retry logic in the express app. However, I would prefer to use Docker's healthcheck to prevent the string of errors when the containers initially spin up. However, when I add a healthcheck in my docker-compose.yml, it hangs for the interval/retry time limit and exits with:

ERROR: for collector  Container "70e7aae49c64" is unhealthy.

ERROR: for server  Container "70e7aae49c64" is unhealthy.
ERROR: Encountered errors while bringing up the project.

It seems that my healthcheck never returns a healthy status, and I'm not entirely sure why. The entirety of my docker-compose.yml:

version: "2.1"
services:
  mongo:
    image: mongo
    volumes:
      - ./data/mongodb/db:/data/db
    ports:
      - "${DB_PORT}:${DB_PORT}"
    healthcheck:
      test: echo 'db.runCommand("ping").ok' | mongo mongo:27017/test --quiet 1
      interval: 10s
      timeout: 10s
      retries: 5
  collector:
    build: ./collector/
    environment:
      - DB_HOST=${DB_HOST}
      - DB_PORT=${DB_PORT}
      - DB_NAME=${DB_NAME}
    volumes:
      - ./collector/:/app
    depends_on:
      mongo:
        condition: service_healthy
  server:
    build: .
    environment:
      - SERVER_PORT=$SERVER_PORT
    volumes:
      - ./server/:/app
    ports:
      - "${SERVER_PORT}:${SERVER_PORT}"
    depends_on:
      mongo:
        condition: service_healthy

For the test, I've also tried:

["CMD", "nc", "-z", "localhost", "27017"] 

And:

["CMD", "bash", "/mongo-healthcheck"]

I've also tried ditching the healthcheck altogether, following the advice of this guy. Everything stands up, but I get the dreaded errors in the output before a successful connection:

collector_1  | MongoDB connection error: MongoNetworkError: failed to connect to server [mongo:27017] on first connect [MongoNetworkError: connect 
ECONNREFUSED 172.21.0.2:27017]
collector_1  | MongoDB connection with retry
collector_1  | MongoDB connection error: MongoNetworkError: failed to connect to server [mongo:27017] on first connect

The ultimate goal is a clean startup output when running the docker-compose up --build. I've also looked into some of the solutions in this question, but I haven't had much luck with wait-for-it either. What's the correct way to wait for Mongo to be up and running before starting the other containers, and achieving a clean startup?

Answer 1

Firstly, I'd suggest to update the docker-compose.yaml file version to at least 3.4 (version: "3.5"), then please add the start_period option to your mongo healthcheck

Note: start_period is only supported for v3.4 and higher of the compose file format.

start period provides initialization time for containers that need time to bootstrap. Probe failure during that period will not be counted towards the maximum number of retries. However, if a health check succeeds during the start period, the container is considered started and all consecutive failures will be counted towards the maximum number of retries.

So it would look something like this:

healthcheck:
  test: echo 'db.runCommand("ping").ok' | mongo mongo:27017/test --quiet
  interval: 10s
  timeout: 10s
  retries: 5
  start_period: 40s