Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Which jmx metric should be used to monitor the status of a connector in kafka connect?

I'm using the following jmx metrics for kafka connect.

like image 303
Jigar Mehta Avatar asked May 11 '18 11:05

Jigar Mehta


2 Answers

Have a look at Connect Monitoring section in the Kafka docs, it lists all the Kafka Connect specific metrics.

For example there are overall metrics for each connector:

  • kafka.connect:type=connector-metrics,connector="{connector}" which contains a connector status (running, failed, etc)

  • kafka.connect:type=connector-task-metrics,connector="{connector}",task="{task}" which contains the status of individual tasks

If you want more than just the status, there are also additional metrics for both sink and source tasks:

  • kafka.connect:type=connector-task-metrics,connector="{connector}",task="{task}"
  • kafka.connect:type=sink-task-metrics,connector="{connector}",task="{task}"
like image 91
Mickael Maison Avatar answered Jan 04 '23 10:01

Mickael Maison


I still don't have enough rep to comment but I can answer...

Elaborating on Mickael's answer, be careful: currently task metrics disappear when a task is in a failed state rather than show up with the FAILED status. A Jira can be found here and PR can be found here

like image 28
bmoe24x Avatar answered Jan 04 '23 10:01

bmoe24x