Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Spring Kafka SSL setup in Spring boot application.yml

I am trying to setup a Spring Boot Application with a Kafka Client to use SSL. I have my keystore.jks and truststore.jks stored on a filesystem(on a docker container) because of this: https://github.com/spring-projects/spring-kafka/issues/710

Here is my application.yml:

spring:
  kafka:
      ssl:
        key-password: pass
        keystore-location: /tmp/kafka.client.keystore.jks
        keystore-password: pass
        truststore-location: /tmp/kafka.client.truststore.jks
        truststore-password: pass

But when I start the application ( in a docker container) it says:

Caused by: java.lang.IllegalStateException: Resource 'class path resource [tmp/kafka.client.keystore.jks]' must be on a file system
[..]
Caused by: java.io.FileNotFoundException: class path resource [tmp/kafka.client.keystore.jks] cannot be resolved to URL because it does not exist

I checked on the container and the .jks are there in /tmp .

I cannot understand how to pass .jks to spring boot.

UPDATE 06/07/2018

This is my dockerfile

FROM openjdk:8-jdk-alpine
VOLUME /tmp
COPY ssl/kafka.client.keystore.jks /tmp
COPY ssl/kafka.client.truststore.jks /tmp
ARG JAR_FILE
ADD ${JAR_FILE} app.jar
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]
like image 953
Justin Avatar asked Jul 05 '18 21:07

Justin


People also ask

How do I connect to Kafka from a Spring Boot application?

Our example application will be a simple Spring Boot application. In order to connect to Kafka, let's add the spring-kafka dependency in our POM file: We'll also be using a Docker Compose file to configure and test the Kafka server setup. Initially, let's do this without any SSL configuration:

What is Kafka SSL configuration?

spring.kafka.ssl.* Kafka SSL configuration is to provide secure communication between producer/consumer and Kafka server. You need to generate key-store and trust-store files and configure the location and password

How do I override the default Kafka consumer and producer?

We can override these defaults using the application.yml property file. Previously we saw how to create a spring kafka consumer and producer which manually configures the Producer and Consumer. In this example we’ll use Spring Boot to automatically configure them for us using sensible defaults.

How to create a Kafka producer service to send messages?

If you want to create a producer service to send messages to a Kafka topic then you need to create two Classes, First, create a KafkaProducerConfig class which uses producer configuration defined in application.yml and define a KafkaTemplate bean which creates an instance of Kafka producer. We will use this instance in our producer service class.


1 Answers

According to discussion and to enable kafka ssl configuration, first need to enable and set ssl properties in consumerFactory

@Bean
public ConsumerFactory<String, ReportingTask> consumerFactory() {
    Map<String, Object> props = new HashMap<>();
    props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
    props.put(ConsumerConfig.GROUP_ID_CONFIG, groupId);
    props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
    props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, JsonSerializable.class);
    props.put(ConsumerConfig.CLIENT_ID_CONFIG, clientId);
    props.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, enableAutoCommit);
    props.put(ConsumerConfig.AUTO_COMMIT_INTERVAL_MS_CONFIG, autoCommitInterval);
    props.put(ConsumerConfig.SESSION_TIMEOUT_MS_CONFIG, sessionTimeout);
    props.put(ConsumerConfig.MAX_POLL_RECORDS_CONFIG, maxRecords);
    props.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, offSet);
    if (sslEnabled) {
        props.put("security.protocol", "SSL");
        props.put("ssl.truststore.location", trustStoreLocation);
        props.put("ssl.truststore.password", trustStorePassword);

        props.put("ssl.key.password", keyStorePassword);
        props.put("ssl.keystore.password", keyStorePassword);
        props.put("ssl.keystore.location", keyStoreLocation);
    }
    return new DefaultKafkaConsumerFactory<>(props, new StringDeserializer(), new JsonDeserializer<>(Task.class));
}

And copy the certificates into docker container

COPY ssl/stage/* /var/lib/kafka/stage/
like image 200
Deadpool Avatar answered Oct 20 '22 06:10

Deadpool