Why __consumer_offsets topic in kafka is not spreading to all the brokers?

Question

I have a 3 zk nodes cluster. And 7 kafka broker nodes.

So when I create any topic then I can set replica factor and no of partitions using command line arguements.

And these partitions are spread to all the 7 brokers.But there is one topic i.e. __consumer_offsets,it is created automatically and it is spreading to only 1 broker(id=0) with 1 replication.

I am aware that I could change the default no of replication factor in config file.

But, this topic is replicated to only one broker. So what is the parameter which could be changed to replicate the partitions to all the brokers.

Answer 1

It sounds really strange to me. I deployed a 3 Kafka brokers cluster and my __consumer_offsets topic has 50 partitions (the default) split across the 3 brokers and having 3 (the default) as replicas. What you are describing could happen when you start with a single Kafka broker, create consumers for reading topics so that the __consumer_offsets is automatically created and it will be just on the broker 0 (the only one). After that, you add new Kafka brokers: without doing anything the __consumer_offsets will still stay on broker 0; you need to use Kafka a manual partition reassignment for doing that as mentioned above.

Answer 2

You will need to use Kafka partition reassignment. This is unfortunately a semi manual process currently: https://kafka.apache.org/documentation/#basic_ops_cluster_expansion