Menu
I was able to successfully set up zookeeper and one kafka broker yesterday. Everything worked as expected. I shut down kafka (ctrl + c) and then zookeeper.
Today I started zookeeper and when I started kafka (bin/kafka-server-start.sh config/server0.properties), I get the following error. I tried various remedies suggested (removing completely my kafka installation and doing it again from scratch). Still I get same error.
[2016-09-28 16:15:55,895] FATAL Fatal error during KafkaServerStartable startup. Prepare to shutdown (kafka.server.KafkaServerStartable) java.lang.RuntimeException: A broker is already registered on the path /brokers/ids/0. This probably indicates that you either have configured a brokerid that is already in use, or else you have shutdown this broker and restarted it faster than the zookeeper timeout so it appears to be re-registering. at kafka.utils.ZkUtils.registerBrokerInZk(ZkUtils.scala:305) at kafka.utils.ZkUtils.registerBrokerInZk(ZkUtils.scala:291) at kafka.server.KafkaHealthcheck.register(KafkaHealthcheck.scala:70) at kafka.server.KafkaHealthcheck.startup(KafkaHealthcheck.scala:51) at kafka.server.KafkaServer.startup(KafkaServer.scala:244) at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:37) at kafka.Kafka$.main(Kafka.scala:67) at kafka.Kafka.main(Kafka.scala) [2016-09-28 16:15:55,896] INFO [Kafka Server 0], shutting down (kafka.server.KafkaServer) All set up in mac
371
If any broker fails, data should not be lost. For fault-tolerance purposes, the partition is replicated and stored in different brokers. If leader brokers fail, then the controller will elects one of the replicas as the leader.
What I did is to create a simple KafkaConsumer and list all the topics with listTopics(). If the connection is success, then you will get something as a return. Otherwise, you will get a TimeoutException . Show activity on this post.
I have faced the same issue while setting up the kafka and zookeeper multinode cluster.
Root Cause :
As zookeeper ensemble takes some time to setup so when we start kafka brokers, all zookeeper nodes might have not joined zookeeper ensemble yet.
So It may be the case that one kafka node connected to a zookeeper node which is not yet in the ensemble.
Another reason can be kafka broker is killed instead of stopped. When you kill your kafka process or it crashes and it does not close its zookeeper connection correctly.
Solution :
To avoid this situation make sure your zookeeper ensemble is up and running before starting kafka brokers.
Add proper waits between zookeeper and kafka broker services start and stop.
104
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
