Can't log from (fluentd) logdriver using service name in compose

Question

I have the following setup in docker:

• Application (httpd)
• Fluentd
• ElasticSearch
• Kibana

The configuration of the logdriver of the application is describing the fluentd container. The logs will be saved in ES and shown in Kibana.

When the logdriver is configured as this, it works:

web:
    image: httpd
    container_name: httpd
    ports:
      - "80:80"
    links:
      - fluentd
    logging:
      driver: "fluentd"
      options:
        fluentd-address: localhost:24224
        tag: httpd.access

And fluentd is mapping its exposed port 24224 on port 24224 of the host.

 fluentd:
    build: ./fluentd
    image: fluentd
    container_name: fluentd
    links:
      - "elasticsearch"
    ports:
      - "24224:24224"

But I don't want to expose my fluentd on the hostnetwork. I want to keep it 'private' inside the docker network (I only want to map the app and kibana on the host network) so like this:

 fluentd:
   build: ./fluentd
   image: fluentd
   container_name: fluentd
   links:
     - "elasticsearch"

The port 24224 is still exposed (in the dockerfile) but it's not mapped on the host network. Now I want change the config of the logdriver of my app: logging: driver: "fluentd" options: fluentd-address: fluentd:24224 tag: httpd.access

So fluentd is the name of the fluentd container and they are in the same network but the app is not able to make a connection with it.

failed to initialize logging driver: dial tcp: lookup fluentd

Is this maybe because the logging option is executed before the 'link'-option in the compose file?

Is there a way to let this work?

Answer 1

This is not possible currently. The docker deamon which handles the log drivers is a process running on the host machine. It is not a service in your network and is therefore unable to resolve servicenames to IP's. See this github issue for more detailed explanations.

You will have to publish a port for this to work.