Menu
I have kafka producer for my java based web application to push messages to Kafka. As per the documentation I could see kafka producer is thread safe. Does it mean that I can have single instance of Kafka producer and use it by different threads ( web requests ) each will open and close the producer in my case. Will this create any issues ? Or Is better to initiate Producers per request ?
873
You can't have multiple consumers that belong to the same group in one thread and you can't have multiple threads safely use the same consumer. One consumer per thread is the rule. To run multiple consumers in the same group in one application, you will need to run each in its own thread.
The producer manages a single background thread that does I/O as well as a TCP connection to each of the brokers it needs to communicate with.
In addition to multiple producers, Kafka is designed for multiple consumers to read any single stream of messages without interfering with each other.
Yes, if you want to have your producer on Server A and your consumer on server B, you are in the right direction. You need to run a Broker on server A to make it work. The other commands are correct.
Yes, KafkaProducer is threadsafe.
Refer to Class KafkaProducer
A Kafka client that publishes records to the Kafka cluster.
The producer is thread safe and should generally be shared among all threads for best performance.
The producer manages a single background thread that does I/O as well as a TCP connection to each of the brokers it needs to communicate with. Failure to close the producer after use will leak these resources.
121
By far the best approach (which is typical of most stateful clients connectors, eg SQL clients, elasticsearch client, etc) is to instantiate a single instance at application start and share it across all threads. It should only be closed on application shutdown.
3
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
