Kafka Application Log Configuration

Question

I'm trying to move the application/Log4J logs for Apache Kafka (not the data logs). This has become exceptionally hard to research as there are many log related terms with regards to Kafka. I've opened the log4j.properties, and found that the logs are written to: ${kafka.logs.dir} however, I'm not sure where to change kafka.logs.dir. We need to change this location due to low disk space on our VMs.

Any help would be much appreciated!

Answer 1

One way to do it is to override the environment property LOG_DIR prior launching the kafka server, for example in a bash script:

#!/bin/sh
export LOG_DIR=/var/log/kafka
echo 'Kafka application logs set to ' $LOG_DIR
./kafka_2.11-1.1.0/bin/kafka-server-start.sh -daemon  /kafka_2.11-1.1.0/config/server.properties

You can also override the log4j.properties with your own see kafka-server-start.sh (snippet from 1.1.0):

if [ "x$KAFKA_LOG4J_OPTS" = "x" ]; then
    export KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:$base_dir/../config/log4j.properties"
fi