How to consume published messages from the kafka (version 0.10) server which was kerberos authorized, for the authentication keytab file is being used.
I tried with the below command but no outputs were shown.
bin/kafka-console-consumer.sh --bootstrap-server :9092 --topic --from-beginning
To enable authentication and authorization on the broker side, you need to perform two steps on each broker: Configure valid credentials. Configure the proper security protocol and authorizer implementation.
The useKeytab value is the full path to the Kerberos keytab file. The principal value is the Kerberos principal, for example user/host@REALM. Here, host is the host of the center for key distribution and REALM is the Kerberos REALM.
Kerberos Authentication. Kerberos is by far the most common option we see being used in the field to secure Kafka clusters. It enables users to use their corporate identities, stored in services like Active Directory, RedHat IPA, and FreeIPA, which simplifies identity management.
Kerberos-enabled clusters can pose some tricky challenges at times. I've had to deal with some of these myself.
If the Kafka Cluster is Kerberos-enabled then you'll need to supply a jaas.conf file with the Kerberos details. Try following these steps(they worked for me):
KafkaClient { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="<path-to-the-keytab-file>" principal="<kafka-principal>"; };
Note: I've assumed that the Kafka principal and the associated keytab is already created. If not, you'll need to create these first.
security.protocol=SASL_PLAINTEXT sasl.kerberos.service.name=kafka
$export KAFKA_OPTS="-Djava.security.auth.login.config=<path-to-jaas.conf>"
$ kafka-console-consumer --topic <topic-name> --from-beginning --bootstrap-server <anybroker>:9092 --consumer.config <consumer.properties>
EDIT - Steps 3 and 4 could be combined just in case there is a preference to keep these as one command in the command history.
I hope this helps.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With