Menu
How can I get alerted when there is a message rate in some topic higher or lower than usual?
352
You can do this using the load generation tools that ship with Kafka, kafka-producer-perf-test, and kafka-consumer-perf-test. The kafka-*-perf-test tools can be used in several ways. In general, it is expected that these tools be used on a test or development cluster.
Instana monitors Kafka's Key Performance Indicators (KPIs) at the individual component and application level for each instance. The Kafka KPIs are used for real-time health monitoring of each instance and cluster. The Service KPIs are used to understand Kafka's performance within the overall environment.
Kafka emits tons of metrics that allow to monitor its usage and health. This includes metrics for incoming message rate, see the Monitoring section in the docs.
Your use case is typically achieved with a tool that handles metrics and is able to fire alerts based on user defined queries.
For example, Prometheus handles that very well. You need to set it up so metrics from Kafka are forwarded to Prometheus (using https://github.com/prometheus/jmx_exporter). As it's a very common use case, you'll find hundreds of articles online that describe how to set that up. The JMX exporter has sample configs for the Kafka JMX patterns, as well.
Once the metrics are in Prometheus you can aggregate them across brokers and set up Alerts.
56
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
