How to use host's network inside Airflow built on Docker Compose?

Question

I want to migrate from old Airflow that is installed locally to a new one. I am trying to run new Airflow trough Docker and Docker Compose. I am using official docker-compose.yml. The problem that I have is that I can't send email using python smtplib and I am confident that the problem is in network settings of my container. I have tried everything from swiss_knights's answer, but I could not make it work. When I run docker compose up I get infinite loop of . prints from services in which I put network_mode: "host".

I would prefere to use hots's network so my codebase migration can be seamless (localhost part of my .py files won't need editing). How should I edit my .yml file?

Linux CentOS 7, Docker 24.0.5 and Docker Compose v2.20.2

I tried:

• adding network_mode: "host" in each service
• adding

network_mode: bridge
    extra_hosts:
      - "host.docker.internal:host-gateway"

to each service

• created custom network as per swiss_knights's answer

Answer 1

The network_mode: host option worked for me.

To do this, I put network_mode: host in all services in airflow docker-compose (redis, postgres and airflow-common) like this:

image: ${AIRFLOW_IMAGE_NAME:-apache/airflow:2.9.1}
network_mode: host
...
image: postgres:13
network_mode: host
...
image: redis:7.2-bookworm
network_mode: host
...

The only thing to consider is that now to connect to postgresql and redis you need to change the connection strings from postgres/redis to 127.0.0.1, more precisely like this:

AIRFLOW__DATABASE__SQL_ALCHEMY_CONN: postgresql+psycopg2://airflow:[email protected]/airflow
AIRFLOW__CELERY__RESULT_BACKEND: db+postgresql://airflow:[email protected]/airflow
AIRFLOW__CELERY__BROKER_URL: redis://:@127.0.0.1:6379/0