Enable SSL for Kafka Clients

Question

Kafka allows clients to connect over SSL. By default SSL is disabled, but I have enabled by referring the below link. http://docs.confluent.io/2.0.0/kafka/ssl.html

After all configuration was done ,Producer/Consumer unable to produce/consume the message.

    [2016-02-29 09:20:49,189] ERROR Error when sending message to topic ssltopic with key: null, value: 2 bytes with error: Failed to update metadata after 60000 ms. (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
sas
[2016-02-29 09:21:16,031] WARN Failed to send SSL Close message  (org.apache.kafka.common.network.SslTransportLayer)
java.io.IOException: Connection reset by peer

Answer 1

The above answer by supermonk clarifies most of the places to check. I faced the similar problem as the OP and the mistake was not in the broker configuration but the client side configuration.
In the official documentation, although they implicitly mentioned to create the client.keystore as step 1, I missed signing the certificate with the CA as done for the server.keystore. This was causing the Kafka broker to refuse the connection from the clients (producer/consumer).

Performing these two steps has eliminated the problem in my case.

keytool -keystore kafka.client.keystore.jks -alias localhost -certreq -file cert-file
openssl x509 -req -CA ca-cert -CAkey ca-key -in cert-file -out cert-signed -days $VALIDITY -CAcreateserial -passin pass:$PASSWORD

keytool -keystore kafka.client.keystore.jks -alias CARoot -import -file ca-cert
keytool -keystore kafka.client.keystore.jks -alias localhost -import -file cert-signed

This will sign the certificate using the CA-cert and add the CARoot as well as signed certificates to the client.keystore.

Reference: Confluent blog on securing Apache Kafka