Firebase: how to efficiently track user/record change history

Question

[Lots of discussion about this on SO, but almost all dealing in SQL, so a lot of it is N/A to my question as far as I could tell]

i'm writing a CRUD app with react/redux, a firebase backend, & AWS lambda functions where necessary.

My users will be interested in seeing what's been changed and when in their records, but I'm struggling with an efficient way to do this.

My current plan is to maintain a collection of "record updates" records with fields like previousVal, changeDate, and userMakingChange. Redux has some great middleware that would make this fairly straightforward in the client-side.

the img is the database schema i had in mind. Firebase stresses very flat data structures, and I want revision histories to load on-demand, separate from their user/record, hence the intermediate tables of revision histories.

So, my question(s):

1. How do revision history systems handle ballooning storage requirements?
2. Is there a less complex way to do this? I assume there has to be an industry 'best-practice' for something like this.

Answer 1

If you're happy to forego Firebase, then Datomic would be a perfect fit for what you're trying to do. It uses the structural sharing techniques from Clojure's persistent data structures to create efficient copies of data, which share as much structure as possible with the original versions.

As far as being able to revert to past versions goes, I'd say that Datomic is probably the industry standard solution. However, the techniques aren't specific to Datomic and with some work, you could implement them with Firebase too.

However, for what it's worth, I'd don't think it's worth going to the trouble until you find your Firebase instance is outgrowing the size limitations. Instead, I'd suggest going with the absolute simplest approach.

Keep your data flat and treat each entity as a list of values. Each time you want to add a new revision, push it onto the end. To revert, pop the required number of values off the stack. It wouldn't be too much work to write some Firebase wrappers which treated all references like this.

Alternatively, if you're feeling super ambitious you could try to combine datascript with ClojureScript's persistent data structures, then serialize them to JSON, then store them in Firebase, then use it as though it was like Datomic.