Menu
I am using kafka-python for accessing Kafka. I try to create a Kafka Producer:
kafka_producer = KafkaProducer(bootstrap_servers=['kafka:9092'])
but this fails with exception kafka.errors.NoBrokersAvailable: NoBrokersAvailable.
I've found out I need to add api_version parameter to the KafkaProducer:
kafka_producer = KafkaProducer(bootstrap_servers=['kafka:9092'],
api_version=(0, 10, 1))
This command works.
The question is: how to determine value of api_version?
kafka-broker-api-versions.sh --bootstrap-server localhost:9092 gives me something, but I am not sure if there's a number I can use. I tried random values like api_version=(20, 2, 1) and it also worked.
755
Determine Kafka version by checking runtime class. Determine Kafka version by checking used library. It is Kafka 2.8. 0 using Scala 2.13.
Kafka APIsThe Admin API to manage and inspect topics, brokers, and other Kafka objects. The Producer API to publish (write) a stream of events to one or more Kafka topics. The Consumer API to subscribe to (read) one or more topics and to process the stream of events produced to them.
With acks = -1, the producer waits for the ack. This ack is sent by the broker as above but only after having the messages replicated to all the replica followers on the other brokers.
If you set an api_version directly, you are telling the kafka-python client to always send api requests using that version. This should work for all broker versions equal to or newer than the client version you have chosen.
This was solved in the discussion in the comments already:
Ideally, it's the version of Kafka you've installed, or the lowest version you're targeting. 0.10.2 is a safe default
That's that point at which any version of newer clients should work - cwiki.apache.org/confluence/display/KAFKA/Compatibility+Matrix
157
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
