Connect to Kafka inside VirtualBox

Question

I'm having troubles connecting from my Host (Windows) to Guest (Linux) where I installed Kafka.

I have set up a VM (with VirtualBox) where I installed Confluent tools. In this VM, I run the command:

confluent start schema-registry

It starts zookeeper, kafka, and schema-registry.

Under this VM, I can run

kafka-console-producer --broker-list localhost:9092 --topic test

and

kafka-console-consumer --bootstrap-server localhost:9092 --topic test

and everything is working fine, I can produce and receive messages.

My goal is however to be able to produce and consume messages from my Host, so I setup this port forwarding rule :

From my Windows, I was expecting this command to work:

bin\windows\kafka-console-producer.bat --broker-list 127.0.0.1:9092 --topic test

But all I get is this :

ERROR Error when sending message to topic test with key: null, value: 3 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for test-0: 1529 ms has passed since batch creation plus linger time 

I've tried many different stuff but still can't find a solution.... Any ideas?

Answer 1

It seems to be a host name gap between Guest and Host. What host name does the Linux Guest have?

When producer/consumer accesses the Kafka broker, the Kafka broker returns its host name for data producer or consumer at default settings. So producers/consumers need to resolve broker's host name to IPAddress.

For broker returning an arbitrary host name, use the advertised.listeners settings.

Details are in the "advertised.listeners" configuration docs.