Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Interactive admin shell for Apache Kafka

Tags:

apache-kafka

I am experimenting with Kafka. I am able to get a basic cluster with Zookeeper and a broker running and produce/consume data. Is there a interactive administration shell/cli for Kafka? I would like to have one view of all the topics , the partitions of a topic, the produce/consume rates, how much data is persisted, etc. I have searched online and looked in the installation. I couldn't find any administration utility.

Thanks,

like image 746
Prakash Shankor Avatar asked Jun 05 '14 19:06

Prakash Shankor


People also ask

Is there a Kafka CLI?

The Kafka CLI is an interactive shell environment that provides you with command-line access for managing your Kafka resources programmatically. You can use the Kafka CLI to type in text commands that perform specific tasks within your Kafka environment.

How do I open a Kafka shell?

Step 1: Start the zookeeper as well as the kafka server initially. Step2: Type the command: 'kafka-console-consumer' on the command line. This will help the user to read the data from the Kafka topic and output it to the standard outputs.


2 Answers

There is Kafka Web Console and Kafka Offset Monitor. Kafka Web Console may not have all what you want but in the future it will include what you're asking for.

like image 139
Claude Avatar answered Oct 01 '22 08:10

Claude


Kafka metrics might be exposed over JMX and explored in jconsole.

On each broker add following line somewhere in kafka-server-start.sh on Linux/MacOS:

export JMX_PORT=9999

or kafka-server-start.bat on Windows:

set JMX_PORT=9999

Note that most JMX metrics expose the state of local broker, so you'd have to connect to all brokers simultaneously to get the whole picture. So, it's more convenient to export necessary JMX metrics to the monitoring system of your choice (Ganglia/Graphite/Nagios/etc.).

like image 24
Wildfire Avatar answered Oct 01 '22 06:10

Wildfire