consumer: Cannot connect to amqp://user:**@localhost:5672//: [Errno 111] Connection refused

Question

I am trying to build my airflow using docker and rabbitMQ. I am using rabbitmq:3-management image. And I am able to access rabbitMQ UI, and API.

In airflow I am building airflow webserver, airflow scheduler, airflow worker and airflow flower. Airflow.cfg file is used to config airflow.

Where I am using broker_url = amqp://user:[email protected]:5672/ and celery_result_backend = amqp://user:[email protected]:5672/

My docker compose file is as follows

version: '3'
services:
  rabbit1:
    image: "rabbitmq:3-management"
    hostname: "rabbit1"
    environment:
      RABBITMQ_ERLANG_COOKIE: "SWQOKODSQALRPCLNMEQG"
      RABBITMQ_DEFAULT_USER: "user"
      RABBITMQ_DEFAULT_PASS: "password"
      RABBITMQ_DEFAULT_VHOST: "/"
    ports:
      - "5672:5672"
      - "15672:15672"

    labels:
      NAME: "rabbitmq1"

  webserver:
    build: "airflow/"
    hostname: "webserver"
    restart: always
    environment:
      - EXECUTOR=Celery
    ports:
      - "8080:8080"
    depends_on:
      - rabbit1
    command:  webserver

  scheduler:
    build: "airflow/"
    hostname: "scheduler"
    restart: always
    environment:
      - EXECUTOR=Celery
    depends_on:
      - webserver
      - flower
      - worker
    command: scheduler

  worker:
    build: "airflow/"
    hostname: "worker"
    restart: always
    depends_on:
      - webserver
    environment:
      - EXECUTOR=Celery
    command: worker

  flower:
    build: "airflow/"
    hostname: "flower"
    restart: always
    environment:
      - EXECUTOR=Celery
    ports:
      - "5555:5555"
    depends_on:
      - rabbit1
      - webserver
      - worker
    command: flower

I am able to build images using docker compose. However, I am not able to connect my airflow scheduler to rabbitMQ. I am getting following error:

consumer: Cannot connect to amqp://user:**@localhost:5672//: [Errno 111] Connection refused.

I have tried using 127.0.0.1 and localhost both.

What I am doing wrong ?

Answer 1

From within your airflow containers, you should be able to connect to the service rabbit1. So all you need to do is to change amqp://user:**@localhost:5672//: to amqp://user:**@rabbit1:5672//: and it should work.

Docker compose creates a default network and attaches services that do not explicitly define a network to it.

You do not need to expose the 5672 & 15672 ports on rabbit1 unless you want to be able to access it from outside the application.

Also, generally it is not recommended to build images inside docker-compose.

Answer 2

I solved this issue by installing rabbitMQ server into my system with command sudo apt install rabbitmq-server.