How to work around the issue of deleting data in an eventstore?
I need to permanently and completely delete some data in order to comply to privacy laws.
I have found these alternatives:
Encrypt the data that you need deleted, and store the encryption key in its own table. When the data needs to be deleted, you then only delete the encryption key.
Use event sourcing on the data that does not need deletion, with reference to a CRUD database for the confidential data that need to be deleted.
Are there any other ways of doing it?
On the technical level, event sourcing can be implemented using dedicated storage systems, as well as general-purpose "NoSQL" and SQL databases. If you're interested in the origins of event sourcing, the articles by Greg Young on event sourcing and CQRS are a great place to start.
How it Works. The fundamental idea of Event Sourcing is that of ensuring every change to the state of an application is captured in an event object, and that these event objects are themselves stored in the sequence they were applied for the same lifetime as the application state itself.
Event Sourcing is a design pattern in which results of business operations are stored as a series of events. It is an alternative way to persist data. In contrast with state-oriented persistence that only keeps the latest version of the entity state, Event Sourcing stores each state change as a separate event.
The event sourcing pattern is typically used with the CQRS pattern to decouple read from write workloads, and optimize for performance, scalability, and security. Data is stored as a series of events, instead of direct updates to data stores.
I did that a month ago. Tried to make it as simple as possible. I just replayed the entire event store, modify event data and finally store the event in a new event store. In other words migration. When everything finished OK I deleted/backup the old store. After that I replayed the new event store against the projections because of the changes.
If you do not have the encryption implemented you have to add it somehow. Like replaying the entire event store.
PS: Just want to mention for other readers that the reasons to change the event store are really limited. Do not do use it except when comply to privacy laws or really nasty bug. If you need to delete user's data you could do one of the two things:
First, change your event handlers to not require the data so that things don't break when you remove it.
Then create a small app to read all your events, and write new events to a new event store without the data you needed deleted.
Test that your system still functions using the new event store; can rehydrate all aggregates, and generate all projections/views/readmodels/whateveryoucallthem.
Delete the old event store.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With