Error while fetching metadata kafka {test=LEADER_NOT_AVAILABLE}?

Question

I get the error after running those simple commands -

I started the Zookeeper and the Kafka servers,

I execute the command:

./kafka-topics --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

and execute the command:

./kafka-console-producer --broker-list localhost:9092 --topic test

I obtain a list of WARN like:

[2019-12-08 21:36:13,024] WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id 37 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)

what did I do wrong? Thanks

Answer 1

If your broker has the auto.create.topics.enable set to true, then this error will be transient and you should be able to produce message without any further error. It happens just because the producer is asking for metadata about the topic it wants to write to but that topic doesn't exist in the cluster and the partition leader (where the producer wants to write) doesn't exist yet. If you retry, the broker will create the topic and the command will work fine.

If the above configuration is set to false, then the broker doesn't create the topic automatically on the first request from a client so you have to create it upfront.

Finally, but it's not your case, the above error could even happen when the topic exist but, for example, the broker which is leader for the specific topic partition is down and a new leader election is in progress.