Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

difference between exactly-once and at-least-once guarantees

I'm studying distributed systems and referring to this old question: stackoverflow link

I really can't understand the difference between exactly-once, at-least-once and at-most-once guarantees, I read these concepts in Kafka, Flink and Storm and Cassandra also. For instance someone says that Flink is better because has exactly-once guarantees while Storm has only at-least-once.

I understand that exactly-once mode is better for latency but at the same time it's worse for fault tolerance right? How can recover a stream if I haven't duplicates? and then... if this is a real problem, why exactly-once guarantee is considered better than others?

Someone can give me better definitions?

like image 755
Akinn Avatar asked May 26 '17 15:05

Akinn


People also ask

What is the difference between at most once Vs at least once vs exactly once?

At-most-once is ideal for applications that need high throughput and low latency due to the fire-and-forget nature. It is the default producer and consumer delivery semantic. At-least-once and exactly-once delivery will require additional configuration.

What does exactly once mean?

Exactly-once as the name suggests, there will be only one and once message delivery. It difficult to achieve in practice. In this case offset needs to be manually managed.

What is the meaning of at least once?

At-least once as the name suggests, message will be delivered atleast once. There is high chance that message will be delivered again as duplicate.

What is at least once delivery?

at-least-once delivery means that for each message handed to the mechanism potentially multiple attempts are made at delivering it, such that at least one succeeds; again, in more casual terms this means that messages may be duplicated but not lost.


1 Answers

Below definitions are quoted from Akka Documentation

at-most-once delivery

means that for each message handed to the mechanism, that message is delivered zero or one times; in more casual terms it means that messages may be lost.

at-least-once delivery

means that for each message handed to the mechanism potentially multiple attempts are made at delivering it, such that at least one succeeds; again, in more casual terms this means that messages may be duplicated but not lost.

exactly-once delivery

means that for each message handed to the mechanism exactly one delivery is made to the recipient; the message can neither be lost nor duplicated.

The first one is the cheapest—highest performance, least implementation overhead—because it can be done in a fire-and-forget fashion without keeping state at the sending end or in the transport mechanism. The second one requires retries to counter transport losses, which means keeping state at the sending end and having an acknowledgement mechanism at the receiving end. The third is most expensive—and has consequently worst performance—because in addition to the second it requires state to be kept at the receiving end in order to filter out duplicate deliveries

like image 138
Amit Kumar Avatar answered Sep 19 '22 19:09

Amit Kumar