kafka-console-producer.sh TimeOutException

Question

I am trying to send messages using the Kafka console producer to a Kafka broker that is running on the same machine. When I run

echo "Hello world" | ./kafka-console-producer.sh --broker-list localhost:9092 --topic test 

I receive the following error message:

[2016-06-29 15:00:44,069] ERROR Error when sending message to topic test with key: null, value: 11 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Batch containing 1 record(s) expired due to timeout while requesting metadata from brokers for test-0

When I check if the Kafka broker is listening on port 9092, it is there and running. How do I debug why cannot the command line producer connect to the broker? Thank you!

Answer 1

I experienced the same problem and it seems like Kafka wrote wrong metadata to zookeeper.

The easiest way to get pass this is to remove the znode in which brokers data is being registered. I.e if you use the defaults, the name should be '/brokers' and the command:

$ zkCli.sh
...
[zk: localhost:2181(CONNECTED) 1] rmr /brokers