Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to add JVM parameters to Apache Kafka?

Tags:

apache-kafka

When configuring authentication for kafka, the document mentioned that JVM parameters need to be added when starting kafka server. like:

-Djava.security.auth.login.config=/etc/kafka/kafka_server_jaas.conf

Since we are using bin/kafka-server-start.sh to start the server, the document didn't mention where to specify the JVM parameters.

Modifying the kafka-server-start.sh or kafka-server-class.sh is not a good idea, then what will be the right way to add the parameter at the start?

like image 455
Jakim Avatar asked Nov 30 '16 14:11

Jakim


People also ask

Does Kafka need JVM?

Kafka uses heap space very carefully and does not require setting heap sizes more than 5GB. Kafka uses page cache memory as a buffer for active writers and readers, so after you specify JVM size (using -Xmx and -Xms Java options), leave the remaining RAM available to the operating system for page caching.

Does Kafka need JDK or JRE?

Zookeeper and Kafka are written in Java so you'll need JDK.

How Kafka is used in Java?

Apache Kafka is an open-source stream-processing software platform which is used to handle the real-time data storage. It works as a broker between two parties, i.e., a sender and a receiver. It can handle about trillions of data events in a day.


1 Answers

I'd recommend to use the KAFKA_OPTS environment variable for this.

This environment variable is recognized by Kafka, and defaults to the empty string (= no settings). See the following code snippet from bin/kafka-run-class.sh in the Kafka source code:

# Generic jvm settings you want to add
if [ -z "$KAFKA_OPTS" ]; then
  KAFKA_OPTS=""
fi

So, for example, you can do:

$ export KAFKA_OPTS="-Djava.security.auth.login.config=/etc/kafka/kafka_server_jaas.conf"
$ bin/kafka-server-start.sh

or

$ KAFKA_OPTS="-Djava.security.auth.login.config=/etc/kafka/kafka_server_jaas.conf" bin/kafka-server-start.sh
like image 126
Michael G. Noll Avatar answered Sep 27 '22 21:09

Michael G. Noll