Event sourcing: handle event schema changing

Question

As a software project built on an event sourcing architecture evolves, the event schema (or event types) are among the things most likely to change over time.

One of the benefits event sourcing architecture offers is that it's able to "replay all events" and build a current state from old events.

What if I need to change the event schema (event types), by adding or removing an attribute, or by changing the semantics of an attribute? The current service implementation won't be able to process old events, because they use the old schema (they do not contain an attribute, or the semantics have changed).

Any ideas for how to handle this situation?

Answer 1

This is been the topic of the research that I have been doing in the past 2 years. We found 5 techniques that you can use to handle schema evolution:

1. Versioned events: never change existing events, always introduce new events.
2. Weak schema: handle missing attributes or superfluous attributes gracefully.
3. Upcasters: transform events at runtime, before the application has to process them.
4. In-place transformation: just change what you need to change in your store.
5. Copy-transform: copy your whole store into a new store.

This is all summarized in our paper "The Dark Side of Event Sourcing" (https://www.movereem.nl/files/2017SANER-eventsourcing.pdf)

We have also researched surrounding areas:

• Pruning of your event store to keep the size maintainable.
• How to keep your read models in sync.
• How DDD can help you to prevent schema evolution.

The last part is not yet public. But you can find slides of a talk I gave yesterday at DDDEurope here: https://speakerdeck.com/overeemm/dddeurope-2018-event-sourcing-after-launch