Docker - library initialization failed - unable to allocate file descriptor table - out of memory

Question

I have been try to run Zookeeper and Kafka on Docker container. I got a lot of errors [error occurred during error reporting , id 0xb] and [Too many errors, abort] in my terminal. And then library initialization failed - unable to allocate file descriptor table - out of memory.

I use the following docker-compose.yml file

version: "3.8"
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:5.5.4
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
  kafka:
    image: confluentinc/cp-kafka:5.5.4
    # If you want to expose these ports outside your dev PC,
    # remove the "127.0.0.1:" prefix
    ports:
      - 127.0.0.1:9092:9092
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1

Answer 1

This problem is caused by having too many file handles available to the program. It increases its soft limit on files to the maximum available, then attempts to allocate memory for each of those file handles. For some systems, with high limits, this causes insufficent memory.

It can be fixed by overriding the command arguments of ExecStart in docker.service

sudo systemctl edit docker

and then enter

[Service]
ExecStart=
ExecStart=/usr/bin/dockerd --default-ulimit nofile=65536:65536 -H fd://

Save and run sudo systemctl daemon-reload and sudo systemctl restart docker.

Answer 2

Try to add ulimits configuration to zookeeper like that

---
version: '3'
services:
  zookeeper:
    image: wurstmeister/zookeeper
    ulimits:
      nofile:
        soft: 65536
        hard: 65536
    ports:
      - "2181:2181"