Menu
I'm trying to move the application/Log4J logs for Apache Kafka (not the data logs). This has become exceptionally hard to research as there are many log related terms with regards to Kafka. I've opened the log4j.properties, and found that the logs are written to: ${kafka.logs.dir} however, I'm not sure where to change kafka.logs.dir. We need to change this location due to low disk space on our VMs.
Any help would be much appreciated!
803
If you open script kafka-server-start or /usr/bin/zookeeper-server-start , you will see at the bottom that it calls kafka-run-class script. And you will see there that it uses LOG_DIR as the folder for the logs of the service (not to be confused with kafka topics data).
Kafka stores partition in segments so that finding some message and deleting them is easy. By default size of a segment is 1 GB. Once a segment is full, new messages produced by producers will be written in new segment.
The Kafka log files are created at the /opt/bitnami/kafka/logs/ directory. The main Kafka log file is created at /opt/bitnami/kafka/logs/server.
One way to do it is to override the environment property LOG_DIR prior launching the kafka server, for example in a bash script:
#!/bin/sh
export LOG_DIR=/var/log/kafka
echo 'Kafka application logs set to ' $LOG_DIR
./kafka_2.11-1.1.0/bin/kafka-server-start.sh -daemon /kafka_2.11-1.1.0/config/server.properties
You can also override the log4j.properties with your own see kafka-server-start.sh (snippet from 1.1.0):
if [ "x$KAFKA_LOG4J_OPTS" = "x" ]; then
export KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:$base_dir/../config/log4j.properties"
fi
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
