Menu
My project is looking at Apache Kafka as a potential replacement for an aging JMS-based messaging approach. In order to make this transition as smooth as possible, it would be ideal if the replacement queuing system (Kafka) had an asynchronous subscription mechanism, similar to our current project's JMS mechanism of using MessageListener and MessageConsumer to subscribe to topics and receive asynchronous notifications. I don't care so much if Kafka doesn't strictly conform to the JMS API, but conversely, I would prefer not to redesign our entire suite of publish-subscribe-notification classes if I don't need to.
I can find all kinds of KafkaConsumer polling examples, but so far have not been able to find any examples with a client being notified of new messages via asynchronous notification.
Does anyone know if the current version of Kafka (0.10.2 as of the time of this post) provides such an API, or am I stuck with trying to rewrite my legacy code using polling?
673
Kafka is a powerful stream processing tool, but it's an asynchronous tool.
The Kafka Producer has a send() method which is asynchronous. Calling the send method adds the record to the output buffer and return right away. The buffer is used to batch records for efficient IO and compression. The Kafka Producer configures acks to control record durability.
Kafka has four core APIs: The Producer API allows an application to publish a stream of records to one or more Kafka topics. The Consumer API allows an application to subscribe to one or more topics and process the stream of records produced to them.
A callback interface that the user can implement to allow code to execute when the request is complete. This callback will generally execute in the background I/O thread so it should be fast.
Kafka clients provides only on-demand pooling mechanism but you can use spring-kafka. It provides MessageListener interface and KafkaListener annotation and similar. See documentation.
106
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
