How can you use TLS for Kafka in Quarkus?

Question

The Kafka guide from Quarkus works nicely when running Kafka locally in Docker. I'm trying to change this sample by replacing the local Kafka service with a hosted Kafka service in the cloud which requires TLS.

Does anyone know how I can configure this? In the Quarkus documentation and the Smallrye documentation I don't see any properties for this.

I'd like to use the Kafka service in the IBM Cloud. Based on the documentation I've tried the following configuration in application.properties:

kafka.bootstrap.servers=broker-0-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-4-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-3-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-5-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-2-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-1-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093
kafka.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="token" password="...";
kafka.sasl.mechanism=PLAIN
kafka.security.protocol=SASL_SSL
kafka.ssl.protocol=TLSv1.2

Update: I've also tried Gunnar's suggestion below, but it doesn't work. When I use the following application.properties ...

mp.messaging.outgoing.generated-price.connector=smallrye-kafka
mp.messaging.outgoing.generated-price.topic=prices
mp.messaging.outgoing.generated-price.value.serializer=org.apache.kafka.common.serialization.IntegerSerializer

mp.messaging.outgoing.generated-price.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="token" password="...";
mp.messaging.outgoing.generated-price.sasl.mechanism=PLAIN
mp.messaging.outgoing.generated-price.security.protocol=SASL_SSL
mp.messaging.outgoing.generated-price.ssl.protocol=TLSv1.2

mp.messaging.incoming.prices.connector=smallrye-kafka
mp.messaging.incoming.prices.topic=prices
mp.messaging.incoming.prices.value.deserializer=org.apache.kafka.common.serialization.IntegerDeserializer

mp.messaging.outgoing.prices.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="token" password="...";
mp.messaging.outgoing.prices.sasl.mechanism=PLAIN
mp.messaging.outgoing.prices.security.protocol=SASL_SSL
mp.messaging.outgoing.prices.ssl.protocol=TLSv1.2

kafka.bootstrap.servers=broker-0-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-4-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-3-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-5-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-2-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-1-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093

... I get an error: javax.enterprise.inject.spi.DeploymentException: java.lang.IllegalArgumentException: Invalid channel configuration - the connector attribute must be set for channel prices at io.quarkus.smallrye.reactivemessaging.runtime.SmallRyeReactiveMessagingLifecycle.onApplicationStart(SmallRyeReactiveMessagingLifecycle.java:22)

Is TLS currently possible for Kafka in Quarkus?

Thanks

Answer 1

Have you tried specifying the relevant properties at the channel level? E.g.

mp.messaging.outgoing.generated-price.connector=smallrye-kafka
mp.messaging.outgoing.generated-price.topic=mytopic
mp.messaging.outgoing.generated-price.ssl.protocol=...
mp.messaging.outgoing.generated-price.ssl.keystore.location=...
mp.messaging.outgoing.generated-price.ssl.keystore.password=...

You also could refer to variables when requiring the same values for multiple topics.