Kafka - Docker - Error when sending message from Host to Container (Batch Expired)

Question

I'm having a difficult time troubleshooting, what should be a simple kafka-issue.

I'm trying to publish a few messages from kafak-console-producer. When I type 'hi' I get:

/opt/kafka/bin/kafka-console-producer.sh --topic test --broker-list 172.17.0.21:9092
hi
[2016-01-25 12:56:19,839] ERROR Error when sending message to topic test with key: null, value: 2 bytes with error: Batch Expired (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)

I don't seem to be able to find any kafka-log4j-logs...

I'm running the kafka server the way it is described in the quick-start

I can create the topic & describe it.

One note which maybe important is that the kafka & zookeeper are running in a docker-container (172.17.0.21) I'm creating the topic and sending messages from the host.

---

Please note the both the (kafka-server) 9092 & (zk) 2181 are open and accessible from the host, I've confirmed this using netstat & telnet.

Answer 1

BTW, you know that you can set that variable from the docker run command, right?

E.g. this is how I start Kafka & ZK in a single container, for my local development environment:

docker run --name st-kafka -p 2181:2181 -p 9092:9092 -e ADVERTISED_HOST=`docker-machine ip machine_name` -e ADVERTISED_PORT=9092 -d spotify/kafka

Then, to create a topic:

docker exec -ti st-kafka bash -c "/opt/kafka_*/bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 4 --topic topic_name"

That's on OS X, which is why I use docker-machine ip MACHINE_NAME.
As you can see, that's Spotify's image. Although there was no v0.9 last time I checked, it's trivial to fork the Dockerfile and adjust for 0.9.

Answer 2

Turned out to be more of a Docker issue...

OK, when running the client kafka-console-consumer.sh I saw that it was trying to connect to the broker using the container's name (which is the first few characters of a GUID).

My host cannot resolve connecting to docker-container using host-name.

Since I had already mapped the ports to my host, I changed the kafak-server config's advertised.host.name to point to my host's IP address.

That way whenever zk is queried for the broker my host's IP address is returned, then through port forwarding I'm connected to the broker in docker.

Answer 3

I am trying with kafka 0.10.0.0 in docker and found below

advertised.host.name DEPRECATED: only used when advertised.listeners or listeners are not set. Use advertised.listeners instead. Hostname to publish to ZooKeeper for clients to use. In IaaS environments, this may need to be different from the interface to which the broker binds. If this is not set, it will use the value for host.name if configured. Otherwise it will use the value returned from java.net.InetAddress.getCanonicalHostName().

so set advertised.listeners=PLAINTEXT://yourhost:9092 will work