Kafka replicates the topic's partitions across the multiple machines for fault-tolerance. Each partition has a leader and one or more number of followers.
I want to know how Kafka chooses the machines that will become the followers of each topic/partition among the possible candidates?
For example, let's say there is 1 topic with 3 partitions {A,B,C} and replication factor is 3. The Kafka is running over 6 machines {1,2,...,6}.
One possible assignment is:
1 2 3 4 5 6
A B C
C A B
B C A
But the following is also possible:
1 2 3 4 5 6
A B C
A B C
A B C
Of course, there are tons of possible assignments.
Can anyone give me an idea how Kafka is doing this?
Thanks
You may look into Kafka sources on github to see how it does replica assignment by default - https://github.com/apache/kafka/blob/0.10.0/core/src/main/scala/kafka/admin/AdminUtils.scala#L47-L106
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With