Menu
In Apache Kafka, each broker has its own configuration file. Some of the config entries, such as broker ID, are evidently unique to each node.
However, others such as topic retention time or maximum message size should be global to the entire cluster.
In case two brokers have conflicting configurations, which value gets precedence? Or am I wrong to assume that some config entries should be global?
570
During a broker outage, all partition replicas on the broker become unavailable, so the affected partitions' availability is determined by the existence and status of their other replicas. If a partition has no additional replicas, the partition becomes unavailable.
The leader handles all the read and writes operations of data for the partitions. The leader and its followers are determined by the zookeeper(discussed later). If the broker holding the leader for the partition fails to serve the data due to any failure, one of its respective ISR replicas will takeover the leadership.
The controller is one of the Kafka brokers that is also responsible for the task of electing partition leaders (in addition to the usual broker functionality).
The Kafka instance (Broker) configurations are kept in the config directory. Go to the config directory. Open the file server. properties.
Kafka does not check that each broker has exactly the same configuration.
That said, as you've pointed, some settings could conflict and if it's the case my guess is that at best a crash or worse undefined behaviour !
There is KIP-226 in progress that addresses some of these issues but if you're to deploy many brokers it's recommended you use some automation (K8s, Mesos) to ensure configuration is consistent across all of them.
173
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
