Event store and optimistic concurrency

Question

Greg Young in his document on CQRS in the "Building an event storage" section, when writing events to the event store he checked for optimistic concurrency. I do not really get why he made that check, can anyone explain to me with a concrete example.

Answer 1

I do not really get why he made that check, can anyone explain to me with a concrete example.

Event stores are supposed to be persistent, in the sense that once you write an event, it will be visible to every subsequent read. So every action in the database should be an append. A useful mental model is to think of a singly linked list.

If a database is going to support more than one thread of execution having write access, then you are faced with the "lost update" problem. Drawn as linked lists, this might look like:

Thread(1) [... <- 69726c3e <- /x.tail] = get(/x)
Thread(2) [... <- 69726c3e <- /x.tail] = get(/x)
Thread(1) set(/x, [ ... <- 69726c3e <- 709726c3 <- /x.tail])
Thread(2) set(/x, [ ... <- 69726c3e <- 83b97195 <- /x.tail])

The history written by thread(2) doesn't include event:709726c3 recorded by thread(1). Thus "lost update".

In a generic database, you typically manage this with transactions: some magic under the covers keeps track of all of your data dependencies, and if the preconditions don't hold when you try to commit your transaction, all of your work is rejected.

But event stores don't use need all of the degrees of freedom that support the general case -- edits to events stored in the database are forbidden, as is changing the dependencies between events.

The only mutable part of the change - which is the only place where we replace overwrite an old value with a new value - is when we change /x.tail

Thread(1) [... <- 69726c3e <- /x.tail] = get(/x)
Thread(2) [... <- 69726c3e <- /x.tail] = get(/x)
Thread(1) set(/x, [ ... <- 69726c3e <- 709726c3 <- /x.tail])
Thread(2) set(/x, [ ... <- 69726c3e <- 83b97195 <- /x.tail])

The problem here is simply that Thread(2) thought 6 <- /x.tail was true, and replaced it with a value that lost event 7. If we change our write from a set to a compare-and-set...

Thread(1) [... <- 69726c3e <- /x.tail] = get(/x)
Thread(2) [... <- 69726c3e <- /x.tail] = get(/x)
Thread(1) compare-and-set(/x, 69726c3e <- /x.tail, [ ... <- 69726c3e <- 709726c3 <- /x.tail])
Thread(2) compare-and-set(/x, 69726c3e <- /x.tail, [ ... <- 69726c3e <- 83b97195 <- /x.tail]) // FAILS

then the data store can detect the conflict and reject the invalid write.

Of course, if the data store sees the actions of the threads in a different order, then the command that fails could change

Thread(1) [... <- 69726c3e <- /x.tail] = get(/x)
Thread(2) [... <- 69726c3e <- /x.tail] = get(/x)
Thread(2) compare-and-set(/x, 69726c3e <- /x.tail, [ ... <- 69726c3e <- 83b97195 <- /x.tail])
Thread(1) compare-and-set(/x, 69726c3e <- /x.tail, [ ... <- 69726c3e <- 709726c3 <- /x.tail]) // FAILS

Put more simply, where set gives us "last writer wins" semantics, compare-and-set gives us "first writer wins", which eliminates the lost update concern.

Answer 2

TLDR; This concurrency check is needed because what events are emitted depends on the previous events. So, if there are other events that are emitted concurrently by another process then the decision must be re-made.

The way that an Event store is used is like this:

1. The old events are loaded from the Eventstream (=a partition in the Eventstore that contains all the events that were generated by an Aggregate instance)
2. The old events are processed/applied by the Aggregate that owns them in the order they were generated
3. The Aggregate, based on the internal state that was build from those events, decides to emit some new events
4. These new events are appended to the Eventstream

So, step 3 depends on the previous events that were generated before this command is executed.

If some events generated in parallel by another process are appended to the same Eventstream then it means that the decision that was made was based on a false premise and thus must be re-taken by repeating from step 1.