Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to build efficient Kafka broker healthcheck?

In my app I will perform some kind of health check of my Kafka cluster.

Currently I make a TopicMetadataRequest to detect dead brokers:

  Future {
    // this will fail if Kafka is unavailable
    consumer.send(new TopicMetadataRequest(Seq("health-check-topic"), 1))
  }

Unfortunately this call produces a huge network traffic, because of Cluster topology/settings.

Is there a better way to check kafka brokers? What I need is something simple like true/false indicator.

like image 361
codejitsu Avatar asked Jul 17 '15 09:07

codejitsu


People also ask

How do you check the health of a Kafka broker?

The kafka-check command performs multiple checks on the health of the cluster. Each subcommand will run a different check. The tool can run on the broker itself or on any other machine, and it will check the health of the entire cluster.

How can I improve my Kafka performance?

Increasing the number of partitions and the number of brokers in a cluster will lead to increased parallelism of message consumption, which in turn improves the throughput of a Kafka cluster; however, the time required to replicate data across replica sets will also increase.

How do I check my Kafka broker port?

Answer : To find the Kafka port number - locate the Kafka server. properties file. Typically the server. properties file will have the information required.

How many Kafka brokers should I have?

Kafka Brokers Connecting to one broker bootstraps a client to the entire Kafka cluster. For failover, you want to start with at least three to five brokers. A Kafka cluster can have, 10, 100, or 1,000 brokers in a cluster if needed.


2 Answers

You can also use Zookeeper API to get the broker list as follows:

ZooKeeper zk = new ZooKeeper(KafkaContextLookupUtil.getZookeeperConnect().getZkConnect(), 10000, null);
    List<String> ids = zk.getChildren("/brokers/ids", false);
    List<Map> brokerList = new ArrayList<>();
    ObjectMapper objectMapper = new ObjectMapper();

    for (String id : ids) {
        Map map = objectMapper.readValue(zk.getData("/brokers/ids/" + id, false, null), Map.class);
        brokerList.add(map);
    }
    return brokerList;
like image 61
Harvinder Singh Avatar answered Oct 11 '22 05:10

Harvinder Singh


If you want to build your own health check, this is a current (January 2020) list of KIPs covering health checks:

  • KIP-143: Controller Health Metrics
  • KIP-188: Add new metrics to support health checks
  • KIP-237: More Controller Health Metrics

Regarding Harvinder Singh's currently accepted answer:

Kafka Manager is great but it's evolving slowly. There's of course Confluent Control Center - a part of Confluent Platform, but you'll need a license for it. Confluent is a company founded by the team that built Apache Kafka. I've heard about akHQ (ex KafkaHQ) (HackerNews story). Here's a list of management consoles maintained on Apache Kafka Confluence page (check URLs there):

  • Kafka Manager - A tool for managing Apache Kafka.
  • kafkat - Simplified command-line administration for Kafka brokers.
  • Kafka Web Console - Displays information about your Kafka cluster including which nodes are up and what topics they host data for.
  • Kafka Offset Monitor - Displays the state of all consumers and how far behind the head of the stream they are.
  • Capillary - Displays the state and deltas of Kafka-based Apache Storm topologies. Supports Kafka >= 0.8. It also provides an API for fetching this information for monitoring purposes.
  • Doctor Kafka - Service for cluster auto healing and workload balancing.
  • Cruise Control - Fully automate the dynamic workload rebalance and self-healing of a Kafka cluster.
  • Burrow - Monitoring companion that provides consumer lag checking as a service without the need for specifying thresholds.
  • Chaperone - An audit system that monitors the completeness and latency of data stream.

If you don't need GUI, there are also:

  • https://github.com/andreas-schroeder/kafka-health-check
  • and its fork https://github.com/ustream/kafka-health-check
like image 39
sm4rk0 Avatar answered Oct 11 '22 04:10

sm4rk0