How to authenticate/authorize a consumer in Kafka for a topic before it consumes the message

Question

In kafka, is there a way to authenticate / authorize a consumer every time a consumer tries to read a message on a topic that it has subscriber to ?

The use case here is that a consumer should be able to present a auth token to the kafka broker and then, broker should be able to validate that token before letting consumer read a message from the topic.

Is the achievable in kafka ?

Answer 1

Kafka provides both pluggable authentication and authorization mechanisms.

For authentication, the process of creating your own logic is described in: Can Kafka be provided with custom LoginModule to support LDAP?

For authorization, you simply need to provide a class that implements the Authorizer interface (https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/security/auth/Authorizer.scala) and set authorizer.class.name to your class in server.properties.

Authentication is only performed when a client connects so if you require validation to happen for every consume action, you'll have to use authorizations.

I suggest you get yourself familiar with Kafka security features and more specifically Authorizations: http://kafka.apache.org/documentation/#security_authz