docker compose networks inside GitHub actions

Question

So, I'm attempting to replicate a flow for setting up my docker stack that I have working well locally, inside a Github action to perform some testing under as close to production / real world scenarios as possible.

However, I'm running into the following issue on the GitHub action workflow which causes a failure:

psycopg2.OperationalError: could not translate host name "db" to address: Temporary failure in name resolution

Essentially, what works in terms of networks locally - doesn't work inside the Github action. The only seeming different is potentially the version of docker I am using within the Github action.

This is my workflow/ci.yml file:

name: perseus/ci

on:
  pull_request:
    branches:
      - main
    paths-ignore:
      - '__pycache__'
      - '.pytest_cache'

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

jobs:
  build:
    name: CI/CD Build & Test w/pytest
    strategy:
      matrix:
        os: [ ubuntu-latest ]

    runs-on: ${{ matrix.os }}

    env:
      PROJECT_NAME: "Perseus FastAPI"
      FIRST_SUPERUSER_EMAIL: ${{ secrets.FIRST_SUPERUSER_EMAIL }}
      FIRST_SUPERUSER_PASSWORD: ${{ secrets.FIRST_SUPERUSER_PASSWORD }}
      POSTGRES_USER: "postgres"
      POSTGRES_PASSWORD: "postgres"
      POSTGRES_SERVER: "db"
      POSTGRES_PORT: "5432"
      POSTGRES_DB: "postgres"
      SENTRY_DSN: ${{ secrets.SENTRY_DSN }}
      SERVER_NAME: "perseus"
      SERVER_HOST: "https://perseus.observerly.com"

    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Setup Environment File
        run: |
          touch .env
          echo PROJECT_NAME=${PROJECT_NAME} > .env
          echo FIRST_SUPERUSER_EMAIL=${FIRST_SUPERUSER_EMAIL} > .env
          echo FIRST_SUPERUSER_PASSWORD=${FIRST_SUPERUSER_PASSWORD} > .env
          echo POSTGRES_USER=${POSTGRES_USER} > .env
          echo POSTGRES_PASSWORD=${POSTGRES_PASSWORD} > .env
          echo POSTGRES_SERVER=${POSTGRES_SERVER} > .env
          echo POSTGRES_PORT=${POSTGRES_PORT} > .env
          echo POSTGRES_DB=${POSTGRES_DB} > .env
          echo SENTRY_DSN=${SENTRY_DSN} > .env
          echo SERVER_NAME=${SERVER_NAME} > .env
          echo SERVER_HOST=${SERVER_HOST} > .env
          cat .env

      - name: Docker Compose Build
        run: docker compose -f local.yml build --build-arg INSTALL_DEV="true"

      - name: Docker Compose Up
        run: docker compose -f local.yml up -d

      - name: Alembic Upgrade Head (Run Migrations)
        run: docker compose -f local.yml exec api alembic upgrade head

      - name: Seed Body (Stars, Galaxies etc) Data
        run: docker compose -f local.yml exec api ./scripts/init_db_seed.sh

Essentially, all steps are working up until the api service needs to talk to the db service, over the network e.g., db:5432

My local.yml file is as follows:

version: '3.8'

services:
  traefik:
    image: traefik:latest
    container_name: traefik_proxy
    restart: always
    security_opt:
      - no-new-privileges:true
    command:
        ## API Settings - https://docs.traefik.io/operations/api/, endpoints - https://docs.traefik.io/operations/api/#endpoints ##
        - --api.insecure=true # <== Enabling insecure api, NOT RECOMMENDED FOR PRODUCTION
        - --api.dashboard=true # <== Enabling the dashboard to view services, middlewares, routers, etc...
        - --api.debug=true # <== Enabling additional endpoints for debugging and profiling
        ## Log Settings (options: ERROR, DEBUG, PANIC, FATAL, WARN, INFO) - https://docs.traefik.io/observability/logs/ ##
        - --log.level=ERROR # <== Setting the level of the logs from traefik
        ## Provider Settings - https://docs.traefik.io/providers/docker/#provider-configuration ##
    labels:
        # Enable traefik on itself to view dashboard and assign subdomain to view it
        - traefik.enable=false
        # Setting the domain for the dashboard
        - traefik.http.routers.api.rule=Host("traefik.docker.localhost")
        # Enabling the api to be a service to access
        - traefik.http.routers.api.service=api@internal
    ports:
        # HTTP  
        - 80:80
        # HTTPS / SSL port
        - 443:443
    volumes:
        # Volume for docker admin
        - /var/run/docker.sock:/var/run/docker.sock:ro
        # Map the static configuration into the container
        - ./traefik/traefik.yml:/etc/traefik/traefik.yml:ro
        # Map the configuration into the container
        - ./traefik/config.yml:/etc/traefik/config.yml:ro
        # Map the certificats into the container
        - ./certs:/etc/certs:ro
    networks:
      - web

  api:
    build: .
    command: uvicorn app.main:app --host 0.0.0.0 --port 5000 --reload --workers 1 --ssl-keyfile "./certs/local-key.pem" --ssl-certfile "./certs/local-cert.pem" --ssl-cert-reqs 1
    container_name: perseus_api
    restart: always
    ports:
      - 8001:5000
    volumes: 
      - .:/app
    depends_on:
      - db
    links:
      - db:db
    env_file:
      - .env
    labels:
      # The following labels define the behavior and rules of the traefik proxy for this container 
      # For more information, see: https://docs.traefik.io/providers/docker/#exposedbydefault
      # Enable this container to be mapped by traefik:
      - traefik.enable=true
      # URL to reach this container:
      - traefik.http.routers.web.rule=Host("perseus.docker.localhost")
      # URL to reach this container for secure traffic:
      - traefik.http.routers.websecured.rule=Host("perseus.docker.localhost")
      # Defining entrypoint for https:
      - traefik.http.routers.websecured.entrypoints=websecured
    networks:
      - web
      - api

  db:
    image: postgres:14-alpine
    container_name: postgres
    volumes:
      - postgres_data:/var/lib/postgresql/data/
      - ./scripts/init_pgtrgm_extension.sql:/docker-entrypoint-initdb.d/init_pgtrgm_extension.sql
    ports:
      - 5432:5432
    env_file:
      - .env
    networks:
      - api

volumes:
    postgres_data:

networks:
  web:
    name: web
  api:
    name: api
    driver: bridge

Are there any networking tips or GitHub action tips that could help me get over this seemingly small hurdle?

I've done as much research as I can into the problem - but I can't seem to see what the solution could be ...

Answer 1

Github actions should allow docker compose networks to work normally. I encountered this error and the problem ended up being that my test app was not fully waiting for my database to be ready. The depends_on is often not sufficient. You need to add a health check and then configure the depends_on to wait until the dependent service is healthy:

Adding a health check to a dependent service:

  postgres:
    image: postgres:15-alpine
    environment:
      POSTGRES_DB: mydb
      POSTGRES_PASSWORD: postgres
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      timeout: 5s
      retries: 5

Waiting until the dependent service is healthy:

  app:
    image: myapp/tester
    build:
      context: ../..
      dockerfile: ./ci/docker/tester.Dockerfile
    command: /usr/bin/python -mpytest myapp/tests/test_postgres.py
    depends_on:
      postgres:
        condition: service_healthy
    environment:
      POSTGRES_HOST: postgres
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
      POSTGRES_DB: mydb