Python KafkaProducer Proceeding to force close the producer since pending requests could not be completed within timeout

Question

Reporting an issue here. The producer threw the errors below. Waited an hour or so and the error went away. But while I had the error I could list topics using using the Kafka tool while the Kafka Python API would not work:

kafka-console-consumer.sh --bootstrap-server localhost:9092 --from-beginning --topic tweets

Here is the error from the python program:

INFO:kafka.producer.kafka:Proceeding to force close the producer since pending requests could not be completed within timeout 0.

Using

from kafka import KafkaProducer
producer = KafkaProducer(bootstrap_servers='localhost:9092')

Answer 1

Its very likely that your producer kernel is shutting down as soon as it sends the message; even before the consumer is able to consume and display the message. Can you try adding sleep(seconds) to your producer script ad try. Worked for me!