I'm trying to create a consumer client in Java. I realized the poll() function is depreciated. What are the alternatives to listen to a topic of Kafka?
My code:
KafkaConsumer< String, UserSegmentPayload > kc = new KafkaConsumer<>(props2);
kc.subscribe(Collections.singletonList(topicName));
while (true) {
ConsumerRecords<String, UserSegmentPayload> records = kc.poll(100);
for (ConsumerRecord<String, UserSegmentPayload> record : records) {
System.out.printf("offset = %d, key = %s, value = %s\n",
record.offset(), record.key(), record.value());
}
}
Kafka-client poll(timeout) is blocking method, so that is not Real Async even if usr reactor-kafka #63.
The Kafka consumer poll() method fetches records in sequential order from a specified topic/partitions. This poll() method is how Kafka clients read data from Kafka. When the poll() method is called, the consumer will fetch records from the last consumed offset.
The poll API is designed to ensure consumer liveness. As long as you continue to call poll, the consumer will stay in the group and continue to receive messages from the partitions it was assigned. Underneath the covers, the consumer sends periodic heartbeats to the server.
The reason poll()
and poll(long)
are deprecated is that they may block indefinitely (even in the second case, where a timeout is specified). The root cause for this behaviour is that an initial metadata update in those methods may block forever (see here). Instead, you should use the poll(Duration)
-method of the KafkaConsumer
. So, in your code, all you have to do is to replace kc.poll(100)
with kc.poll(Duration.ofMillis(100))
.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With