Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Error creating Kafka Topics- Replication factor larger than available brokers

Tags:

apache-kafka

I am trying to create topics in Kafka by following the guide on Apache Kafka website through command line. While running the command:

bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

I get the error as follows:

Error while executing topic command replication factor: 1 larger than available
brokers: 0
kafka.admin.AdminOperationException: replication factor: 1 larger than available
 brokers: 0
        at kafka.admin.AdminUtils$.assignReplicasToBrokers(AdminUtils.scala:70)
        at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:171)
        at kafka.admin.TopicCommand$.createTopic(TopicCommand.scala:93)
        at kafka.admin.TopicCommand$.main(TopicCommand.scala:55)
        at kafka.admin.TopicCommand.main(TopicCommand.scala)

[2015-10-13 14:25:38,607] INFO Processed session termination for sessionid: 0x15
06056566d0004 (org.apache.zookeeper.server.PrepRequestProcessor)
[2015-10-13 14:25:38,608] INFO Closed socket connection for client /127.0.0.1:33
391 which had sessionid 0x1506056566d0004 (org.apache.zookeeper.server.NIOServer
Cnxn)

How can I resolve the issue? Any good documentation to get started with this? Thanks!!

like image 343
Nicole Avatar asked Oct 13 '15 09:10

Nicole


People also ask

Can replication factor be greater than available brokers?

The thing is, you cannot create the Kafka topic with replication factor greater than your broker.

How do you increase Kafka topic replication factor?

Increasing the replication factor can be done via the kafka-reassign-partitions tool. Specify the extra replicas in the custom reassignment json file and use it with the --execute option to increase the replication factor of the specified partitions.

What is the maximum value for replication factor of a Kafka topic?

A broker can only host a single replica for a partition. So if your cluster has 3 brokers, the maximum replication factor you can have is 3.

Can we decrease the replication factor in Kafka?

Kafka does not currently support reducing the number of partitions for a topic or changing the replication factor.


2 Answers

Seems like your server not started , check logs for issue

Kill process

sudo fuser -k 2181/tcp 

run zookeeper

bin/zookeeper-server-start.sh config/zookeeper.properties 

Run Kafka

bin/kafka-server-start.sh config/server.properties  '  
like image 107
vaquar khan Avatar answered Sep 20 '22 15:09

vaquar khan


If I were you I would try this code:

bin/kafka-topics.sh --create --zookeeper localhost:2181/kafka --replication-factor 1 --partitions 1 --topic test

If you are using a cluster where zookeeper be distributed in 3 nodes you should substitue localhost:2181/kafka for direccion1:2181,direction2:2181,direction3:2181/kafka

like image 43
Anxo P Avatar answered Sep 17 '22 15:09

Anxo P