Menu
The github examples page for the Confluent Kafka library lists two methods, namely poll and consume. What is the difference between the two.
I did look at the Consumer implementation in the Confluent Kafka library here , and feel they're functionally the same, and differ only in terms of what they return.
Poll() calls consume() to see if there is a message ready to be picked up, and if yes, invokes the OnMessage event. Whereas, consume, saves the message in one of it's parameters, and returns a boolean. I feel difference is in implementation, and functionally they're the same https://github.com/confluentinc/confluent-kafka-dotnet/blob/master/src/Confluent.Kafka/Consumer.cs
850
The poll() method is the function a Kafka consumer calls to retrieve records from a given topic. When calling the poll() method, consumers provide a timeout argument. This is the maximum amount of time to wait for records to process before returning.
So the max. poll. records control the number of messages read at one poll. This allows us to tune the consumption based on the number of messages to be processed without timing out.
When a consumer wants to join a group, it sends a request to the coordinator. The first consumer to participate in a group becomes a leader. All other consumers joining later becomes the members of the group. So, we have two actors, A coordinator, and a group leader.
See this answer for more details. max.poll.interval.ms default value is five minutes, so if your consumerRecords. forEach takes longer than that your consumer will be considered dead.
You are right. https://github.com/confluentinc/confluent-kafka-dotnet/blob/3f48e8944242abf631ea8d2cd5698f6e149e0731/src/Confluent.Kafka/Consumer.cs#L1147
Poll is the same as Consume, just expose message with different semantic. Consume will return message in out parameter whereas Poll return null and will fire OnMessage event.
Edit: in v1 there is only Consume call left, no more confusion.
90
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
