Can't figure out setting for inter.broker.listener.name in Kafka with SSL

Question

I am attempting to configure Kafka nodes with SSL (TLS) inter-nodes and between nodes and clients but run into configuration problems. Kafka version is 2.3.0. My relevant settings are:

      - KAFKA_BROKER_ID=1
      - ALLOW_PLAINTEXT_LISTENER=yes
      - KAFKA_LISTENERS=LISTENER_INTERNAL://kafka1:9092,LISTENER_EXTERNAL://kafka1:29092
      - KAFKA_ADVERTISED_LISTENERS=LISTENER_INTERNAL://kafka1:9092,LISTENER_EXTERNAL://localhost:29091
      - KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=LISTENER_INTERNAL:SSL,LISTENER_EXTERNAL:SSL
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper1:2181,zookeeper2:2181,zookeeper3:2181
      - KAFKA_AUTO_CREATE_TOPICS_ENABLE=false
      - KAFKA_SSL_TRUSTSTORE_LOCATION=/var/private/ssl/server.truststore.jks
      - KAFKA_SSL_TRUSTSTORE_PASSWORD=changeit
      - KAFKA_SSL_KEYSTORE_LOCATION=/var/private/ssl/server.keystore.jks
      - KAFKA_SSL_KEYSTORE_PASSWORD=changeit
      - KAFKA_SSL_KEY_PASSWORD=changeit
      - KAFKA_SECURITY_INTER_BROKER_PROTOCOL=SSL
      - KAFKA_SSL_CLIENT_AUTH=required

FYI, for simplicity I copied the settings from the docker-compose file that instantiates the Kafka container. The env vars map 1:1 to properties in server.properties. During container start, these settings are applied to the server.properties file.

When I start with this configuration, I receive the following error message:

java.lang.IllegalArgumentException: requirement failed: inter.broker.listener.name must be a listener name defined in advertised.listeners. The valid options based on currently configured listeners are LISTENER_INTERNAL,LISTENER_EXTERNAL

When I set the inter.broker.listener.name property to either INTERNAL_LISTENER, SSL, null or empty string, I receive instead this error message:

org.apache.kafka.common.config.ConfigException: Only one of inter.broker.listener.name and security.inter.broker.protocol should be set.

I have spent a few hours on this issue. I have compared my settings to those few examples on the web that are supposed to demonstrate Kafka with SSL configuration.

Any idea?

Answer 1

I finally figured out how to have multiple SSL listeners. I'll document this here in case someone else runs into the same issue, since working examples of multiple SSL listeners seem to be rare to non-existent. Below is my working configuration (only showing the relevant properties passed through from docker-compose):

ALLOW_PLAINTEXT_LISTENER=no
KAFKA_LISTENERS=ISSL://kafka1:9092,OSSL://kafka1:29092
KAFKA_ADVERTISED_LISTENERS=ISSL://kafka1:9092,OSSL://localhost:29092
KAFKA_INTER_BROKER_LISTENER_NAME=ISSL
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=ISSL:SSL,OSSL:SSL
KAFKA_SSL_CLIENT_AUTH=required

The key to this was to NOT configure the KAFKA_SECURITY_INTER_BROKER_PROTOCOL as it is mutually exclusive with the KAFKA_INTER_BROKER_LISTENER_NAME key.

In case of multiple listeners, it seems that the combination of KAFKA_LISTENER_SECURITY_PROTOCOL_MAP and KAFKA_INTER_BROKER_LISTENER_NAME is what is required.