Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Connecting to RabbitMQ container with docker-compose

I want to run RabbitMQ in one container, and a worker process in another. The worker process needs to access RabbitMQ.

I'd like these to be managed through docker-compose.

This is my docker-compose.yml file so far:

version: "3"

services:

  rabbitmq:
    image: rabbitmq
    command: rabbitmq-server
    expose:
      - "5672"
      - "15672"

  worker:
    build: ./worker
    depends_on:
      - rabbitmq
    # Allow access to docker daemon
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

So I've exposed the RabbitMQ ports. The worker process accesses RabbitMQ using the following URL:

amqp://guest:guest@rabbitmq:5672/

Which is what they use in the official tutorial, but localhost has been swapped for rabbitmq, since the the containers should be discoverable with a hostname identical to the container name:

By default Compose sets up a single network for your app. Each container for a service joins the default network and is both reachable by other containers on that network, and discoverable by them at a hostname identical to the container name.

Whenever I run this, I get an connection refused error:

Recreating ci_rabbitmq_1 ... done                                                                                                                                                    
Recreating ci_worker_1   ... done                                                                                                                                                    
Attaching to ci_rabbitmq_1, ci_worker_1                                                                                                                                              
worker_1    | dial tcp 127.0.0.1:5672: connect: connection refused                                                                                                                   
ci_worker_1 exited with code 1        

I find this interesting because it's using the IP 127.0.0.1 which (I think) is localhost, even though I specified rabbitmq as the hostname. I'm not an expert on docker networking, so maybe this is desired.

I'm happy to supply more information if needed!


Edit

There is an almost identical question here. I think I need to wait until rabbitmq is up and running before starting worker. I tried doing this with a healthcheck:

version: "2.1"

services:

  rabbitmq:
    image: rabbitmq
    command: rabbitmq-server
    expose:
      - "5672"
      - "15672"
    healthcheck:
      test: [ "CMD", "nc", "-z", "localhost", "5672" ]
      interval: 10s
      timeout: 10s
      retries: 5

  worker:
    build: .
    depends_on:
      rabbitmq:
        condition: service_healthy

(Note the different version). This doesn't work, however - it will always fail as not-healthy.

like image 840
haz Avatar asked Oct 28 '18 12:10

haz


People also ask

How do I connect a RabbitMQ docker container?

Open a terminal, navigate to your rabbitmq-go folder and run docker-compose up . This command will pull the rabbitmq:3-management-alpine image, create the container rabbitmq and start the service and webUI. You should see something like this: Once you see this, open your browser and head over to http://localhost:15672.

Can't connect to RabbitMQ?

Make sure the node is running using rabbitmq-diagnostics status. Verify config file is correctly placed and has correct syntax/structure. Inspect listeners using rabbitmq-diagnostics listeners or the listeners section in rabbitmq-diagnostics status. Inspect effective configuration using rabbitmq-diagnostics environment.

What is the default port for RabbitMQ?

By default, RabbitMQ will listen on port 5672 on all available interfaces.

How do I change my RabbitMQ docker username and password?

If you wish to change the default username and password of guest / guest , you can do so with the RABBITMQ_DEFAULT_USER and RABBITMQ_DEFAULT_PASS environmental variables. These variables were available previously in the docker-specific entrypoint shell script but are now available in RabbitMQ directly.


4 Answers

Aha! I fixed it. @Ijaz was totally correct - the RabbitMQ service takes a while to start, and my worker tries to connect before it's running.

I tried using a delay, but this failed when the RabbitMQ took longer than usual.

This is also indicative of a larger architectural problem - what happens if the queuing service (RabbitMQ in my case) goes offline during production? Right now, my entire site fails. There needs to be some built-in redundancy and polling.

As described this this related answer, we can use healthchecks in docker-compose 3+:

version: "3"

services:

  rabbitmq:
    image: rabbitmq
    command: rabbitmq-server
    expose:
      - 5672
      - 15672
    healthcheck:
      test: [ "CMD", "nc", "-z", "localhost", "5672" ]
      interval: 5s
      timeout: 15s
      retries: 1

  worker:
    image: worker
    restart: on-failure
    depends_on:
      - rabbitmq

Now, the worker container will restart a few times while the rabbitmq container stays unhealthy. rabbitmq immediately becomes healthy when nc -z localhost 5672 succeeds - i.e. when the queuing is live!

like image 152
haz Avatar answered Oct 23 '22 22:10

haz


Here is the correct working example :

    version: "3.8"

    services:

    rabbitmq:
        image: rabbitmq:3.7.28-management
        #container_name: rabbitmq
        volumes:
            - ./etc/rabbitmq/conf:/etc/rabbitmq/
            - ./etc/rabbitmq/data/:/var/lib/rabbitmq/
            - ./etc/rabbitmq/logs/:/var/log/rabbitmq/
        environment:
            RABBITMQ_ERLANG_COOKIE: ${RABBITMQ_ERLANG_COOKIE:-secret_cookie}
            RABBITMQ_DEFAULT_USER: ${RABBITMQ_DEFAULT_USER:-admin}
            RABBITMQ_DEFAULT_PASS: ${RABBITMQ_DEFAULT_PASS:-admin}
        ports:
            - 5672:5672    #amqp
            - 15672:15672  #http
            - 15692:15692  #prometheus
        healthcheck:
        test: [ "CMD", "rabbitmqctl", "status"]
        interval: 5s
        timeout: 20s
        retries: 5

    mysql:
        image: mysql
        restart: always
        volumes:
        - ./etc/mysql/data:/var/lib/mysql
        - ./etc/mysql/scripts:/docker-entrypoint-initdb.d
        environment:
        MYSQL_ROOT_PASSWORD: root
        MYSQL_DATABASE: mysqldb
        MYSQL_USER: ${MYSQL_DEFAULT_USER:-testuser}
        MYSQL_PASSWORD: ${MYSQL_DEFAULT_PASSWORD:-testuser}
        ports:
        - "3306:3306"
        healthcheck:
        test: ["CMD", "mysqladmin" ,"ping", "-h", "localhost"]
        timeout: 20s
        retries: 10

    trigger-batch-process-job:
        build: .
        environment:
        - RMQ_USER=${RABBITMQ_DEFAULT_USER:-admin}
        - RMQ_PASS=${RABBITMQ_DEFAULT_PASS:-admin}
        - RMQ_HOST=${RABBITMQ_DEFAULT_HOST:-rabbitmq}
        - RMQ_PORT=${RABBITMQ_DEFAULT_PORT:-5672}
        - DB_USER=${MYSQL_DEFAULT_USER:-testuser}
        - DB_PASS=${MYSQL_DEFAULT_PASSWORD:-testuser}
        - DB_SERVER=mysql
        - DB_NAME=mysqldb
        - DB_PORT=3306
        depends_on:
        mysql:
            condition: service_healthy
        rabbitmq:
            condition: service_healthy
like image 44
craftsmannadeem Avatar answered Oct 23 '22 21:10

craftsmannadeem


Maybe you dont need to expose/map the ports on the host if you are just accessing the service from another container.

From the documentation:

Expose Expose ports without publishing them to the host machine - they’ll only be accessible to linked services. Only the internal port can be specified.

expose:
 - "3000"
 - "8000"

So it should be like this:

version: "3"

services:

  rabbitmq:
    image: rabbitmq
    command: rabbitmq-server
    expose:
      - "5672"
      - "15672"

  worker:
    build: ./worker
    depends_on:
      - rabbitmq
    # Allow access to docker daemon
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

also make sure to connect to rabitmq only when its ready to server on port.

like image 3
Ijaz Ahmad Avatar answered Oct 23 '22 21:10

Ijaz Ahmad


Most clean way for docker compose v3.8

version: "3.8"

services:

  worker:
    build: ./worker
    rabbitmq:
        condition: service_healthy

  rabbitmq:
    image: library/rabbitmq
    ports:
    - 5671:5671
    - 5672:5672
    healthcheck:
      test: [ "CMD", "nc", "-z", "localhost", "5672" ]
      interval: 5s
      timeout: 10s
      retries: 3
like image 2
Amin Shojaei Avatar answered Oct 23 '22 20:10

Amin Shojaei