Securing access to REST API of Kafka Connect

Question

The REST API for Kafka Connect is not secured and authenticated. Since its not authenticated, the configuration for a connector or Tasks are easily accessible by anyone. Since these configurations may contain about how to access the Source System [in case of SourceConnector] and destination system [in case of SinkConnector], Is there a standard way to restrict access to these APIs?

Answer 1

In Kafka 2.1.0, there is possibility to configure http basic authentication for REST interface of Kafka Connect without writing any custom code.

This became real due to implementation of REST extensions mechanism (see KIP-285).

Shortly, configuration procedure as follows:

1. Add extension class to worker configuration file:

rest.extension.classes = org.apache.kafka.connect.rest.basic.auth.extension.BasicAuthSecurityRestExtension

2. Create JAAS config file (i.e. connect_jaas.conf) for application name 'KafkaConnect':

KafkaConnect {
   org.apache.kafka.connect.rest.basic.auth.extension.PropertyFileLoginModule required
             file="/your/path/rest-credentials.properties";
};

3. Create rest-credentials.properties file in above-mentioned directory:

user=password

4. Finally, inform java about you JAAS config file, for example, by adding command-line property to java:

-Djava.security.auth.login.config=/your/path/connect_jaas.conf 

After restarting Kafka Connect, you will be unable to use REST API without basic authentication.

Please keep in mind that used classes are rather examples than production-ready features.

Links:

• Connect configuratin
• BasicAuthSecurityRestExtension
• JaasBasicAuthFilter
• PropertyFileLoginModule

Answer 2

Now you are able to enable certificate based authentication for client access to the REST API of Kafka Connect. An example here https://github.com/sudar-path/kc-rest-mtls

Answer 3

As of Kafka 1.1.0, you can set up SSL and SSL client authentication for the Kafka Connect REST API. See KIP-208 for the details.