"kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection" ONLY DURING LISTING TOPICS

Question

I found few questions with similar topic but different context: I can connect to create a Topic but I can't list the topics because I got the error mentioned below (as far as I could see, people were facing issues for basic connecting while I am getting only for listing the topic list).

In case it matters, here is my docker-compose.yml:

version: "3"
services:
    zookeeper:
        image: wurstmeister/zookeeper

    kafka:
        image: wurstmeister/kafka
        ports:
            - "9092:9092"
        environment:
            KAFKA_ADVERTISED_HOST_NAME: "localhost"
            KAFKA_ZOOKEEPER_CONNECT: "zookeeper:2181"

My console:

bash-4.4# kafka-topics.sh --create --zookeeper zookeeper:2181 --replication-factor 1 --partitions 1 --topic test2
Created topic test2.
bash-4.4# kafka-topics.sh --list --zookeeper localhost:2181
Exception in thread "main" kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING
        at kafka.zookeeper.ZooKeeperClient.$anonfun$waitUntilConnected$3(ZooKeeperClient.scala:259)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
        at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:253)
        at kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:255)
        at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:113)
        at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1858)
        at kafka.admin.TopicCommand$ZookeeperTopicService$.apply(TopicCommand.scala:321)
        at kafka.admin.TopicCommand$.main(TopicCommand.scala:54)
        at kafka.admin.TopicCommand.main(TopicCommand.scala)

---

bash-4.4# kafka-topics.sh --create --zookeeper zookeeper:2181 --replication-factor 1 --partitions 1 --topic test3
Created topic test3.
bash-4.4# kafka-topics.sh --list --zookeeper localhost:2181
Exception in thread "main" kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING
        at kafka.zookeeper.ZooKeeperClient.$anonfun$waitUntilConnected$3(ZooKeeperClient.scala:259)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
        at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:253)
        at kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:255)
        at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:113)
        at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1858)
        at kafka.admin.TopicCommand$ZookeeperTopicService$.apply(TopicCommand.scala:321)
        at kafka.admin.TopicCommand$.main(TopicCommand.scala:54)
        at kafka.admin.TopicCommand.main(TopicCommand.scala)

Edited

Future readers may be found useful how I could list all topics straight from my Docker Kafka Container without logging in my Docker Zookeper Container (https://stackoverflow.com/a/56595227/4148175)

C:\Users\>docker exec -it test1_kafka_1 bash

bash-4.4# kafka-topics.sh --list --bootstrap-server localhost:9092
__consumer_offsets
test2
test3
test_topic
bash-4.4# 

Answer 1

--zookeeper zookeeper:2181 seems to have worked fine

--zookeeper localhost:2181 will always fail inside the kafka container because it's not running a zookeeper server

---

I could list all topics straight from my Docker Kafka Container without logging in my Docker Zookeper Container

That's right. Ideally, you should never enter the Zookeeper container. Latest kafka versions support using --bootstrap-server instead, so you could use kafka:9092 or localhost:9092 from the kafka container