Menu
I am working on kafka connectors and by time my connectors are increasing. So my log file is getting really messy, I was wondering if it is possible to have separate log file for each connector.
786
The Kafka Connect Log4j properties file is located in the Confluent Platform installation directory path etc/kafka/connect-log4j. properties .
If you open script kafka-server-start or /usr/bin/zookeeper-server-start , you will see at the bottom that it calls kafka-run-class script. And you will see there that it uses LOG_DIR as the folder for the logs of the service (not to be confused with kafka topics data). Show activity on this post.
A reference configuration for distributed mode can be found at $CONFLUENT_HOME/etc/kafka/connect-distributed.
x. The following components in Apache Kafka use Log4j-v1. 2.17 : broker, controller, zookeeper, connect, mirrormaker and tools. Clients may also be configured to use Log4j-v1.
you can use grep command to view only the required logs.
command: tail /var/log/kafka/connect.log -f | grep -n 'phrase to search'
your path for log file could be different.
75
Whatever you are trying to achieve is not possible as of current Kafka-connect implementation. But there is a KIP under discussion which may help you when it's accepted and implemented. https://cwiki.apache.org/confluence/display/KAFKA/KIP-449%3A+Add+connector+contexts+to+Connect+worker+logs
30
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
