how to change Kafka broker list ip

Question

I have 3 Kafka brokers running in a isolated network region, my client can not connect them directly, so I have to use a VIP(virtual ip) to connect the brokers.

For example:

my brokers' IP are: 10.5.1.5, 10.5.1.6, 10.5.1.7,

my VIPs' ip are: 200.100.1.5, 200.100.1.6, 200.100.1.7, they one to one paired.

So when I indicate the bootstrap list as 200.100.1.5, the cluster response me the mixed VIPs and Broker ips, such as: 10.5.1.5, 10.5.1.6, 200.100.1.5, 200.100.1.6 ..., then the connection failed, because my program can not reach broker's ip, only can reach VIPs.

My current configuration as following, it responses both IP and VIP:

listeners=INTERNAL://:9092,EXTERNAL_PLAINTEXT://:8080

advertised.listeners=EXTERNAL_PLAINTEXT://200.100.1.5:8080,INTERNAL://10.5.1.5:9092

listener.security.protocol.map=INTERNAL:PLAINTEXT,EXTERNAL_PLAINTEXT:PLAINTEXT

inter.broker.listener.name=INTERNAL

How can I let Kafka only response the VIP list please.

I've got the answer, it could be the following:

advertised.listeners=PLAINTEXT://200.100.1.5:8080
listeners=PLAINTEXT://10.5.1.5:9092
And remove the listener.security and inter.broker.

Answer 1

You can use the broker setting called advertised.listeners to tell your brokers to include a different IP/hostname in their response to clients.

advertised.listeners:

Listeners to publish to ZooKeeper for clients to use, if different than the listeners config property. In IaaS environments, this may need to be different from the interface to which the broker binds. If this is not set, the value for listeners will be used. Unlike listeners it is not valid to advertise the 0.0.0.0 meta-address.

In your example, for the first broker you can have:

advertised.listeners=PLAINTEXT://200.100.1.5:9092
listeners=PLAINTEXT://10.5.1.5:9092