Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to know the broker that is the active controller?

Tags:

apache-kafka

Other than using JMX is there any other way to know, whether a broker is an ActiveController?

I know that the cluster generates a metric kafka.controller:type=KafkaController,name=ActiveControllerCount, but I cannot find which broker is the active controller.

Is it necessary to write a JMX client to know it or is there another (better?) way?

like image 518
Suhas Chikkanna Avatar asked Jan 16 '18 18:01

Suhas Chikkanna


People also ask

How do I know which Kafka controller I have?

The recommended way to detect if a given broker is the controller is via the kafka. controller:type=KafkaController,name=ActiveControllerCount metric.

What is controller broker?

Controller Broker (KafkaController) is a Kafka service that runs on every broker in a Kafka cluster, but only one can be active (elected) at any point in time. The process of promoting a broker to be the active controller is called Kafka Controller Election.

What is active controller in Kafka?

Active controller In a Kafka cluster, one of the brokers serves as the controller, which is responsible for managing the states of partitions and replicas and for performing administrative tasks like reassigning partitions. At any given time there is only one controller broker in your cluster.

How do I get a list of brokers in Kafka?

There are 2 ways to get the list of available brokers in a Kafka cluster. Both with the help of scripts from zookeeper. Zookeeper manages the leader election and other coordination things for a Kafka cluster. So Zookeeper has a list of all the Kafka brokers in the cluster.


2 Answers

You can find the active controller using the zookeeper-shell tool as follows:

./bin/zookeeper-shell.sh [ZK_IP] get /controller
like image 115
Mickael Maison Avatar answered Oct 10 '22 14:10

Mickael Maison


Perhaps easier way, since you don't need to have kafka and it's zookeeper-shell.sh, is using command line to connect to zookeeper's client port with:

nc [zookeeper_ip] [zookeeper_port]

or

telnet [zookeeper_ip] [zookeeper_port]

and then executing

dump

It will print sessions with ephemeral nodes and one can see which broker is a controller by finding session that contains /controller path

For example, a dump result like this:

Sessions with Ephemerals (3):
0x266542cfaa90000:
    /brokers/ids/1
0x166542cfe670001:
    /brokers/ids/3
0x166542cfe670000:
    /controller
    /brokers/ids/2 

means that controller is a broker whose broker_id is 2.

like image 23
sowieso-fruehling Avatar answered Oct 10 '22 16:10

sowieso-fruehling