Is operation in raft log entry supposed to be idempotent?

Question

In raft, when a node restart, it try to redo all the log entries to catch up the state. But if node goes down again in recovery phase, node would do some op twice. These twice redo op will violate state machine if ops are not idempotent.

According to the description above, my question is, is that necessary to make ops idempotent in a system using raft in practice?

Answer 1

In my experience Raft, Paxos, and friends are used to implement distributed state machines and databases (which is a distributed state machine). When viewed in this context you really, really, really want the entries to be idempotent; otherwise your state machines will diverge sooner than later.

You want idempotency even if you were using non-ordered queues---like rabbit-mq or ActiveMQ---because their guarantees are at best at least once delivery.

Sure, there's nothing preventing you from having non-idempotent entries. Except, perhaps a design review. In short, I cannot think of one scenario where a queue is better served with non-idempotent entries. Not one.