Firebase Commit/Rollback for complex writes

Question

I'm writing a financial app with Firebase and for an receipt to be submitted, a number of other objects also need to be updated. For the data to be valid, all data updates need to be completed successfully. If there's an error in one of the writes, all updates must be rolled back.

For example:

If the user submits a receipt, the receipt object must be updated as well as an invoice object as well as other general ledger objects.

If the update started but the user lost internet connection half way through, all changes should be rolled back.

What's the best way to achieve this in Firebase?

Answer 1

First, let's chat for a minute about why someone might want to do commit/rollback on multiple data paths...

Do you need this?

Generally, you do not need this if:

• you are not writing with high concurrency (hundreds of write opes per minute to the SAME record by DIFFERENT users)
• your dependencies are straightforward (B depends on A, and C depends on A, but A does not depend on B or C)
• your data can be merged into a single path

Developers are a bit too worried about orphaned records appearing in their data. The chance of a web socket failing between one write and the other is probably trivial and somewhere on the order of collisions between timestamp based IDs. That’s not to say it’s impossible, but it's generally low consequency, highly unlikely, and shouldn’t be your primary concern.

Also, orphans are extremely easy to clean up with a script or even just by typing a few lines of code into the JS console. So again, they tend to be very low consequence.

What can you do instead of this?

Put all the data that must be written atomically into a single path. Then you can write it as a single set or a transaction if necessary.

Or in the case where one record is the primary and the others depend on this, simply write the primary first, then write the others in the callback. Add security rules to enforce this, so that the primary record always exists before the others are allowed to write.

If you are denormalizing data simply to make it easy and fast to iterate (e.g. to obtain a list of names for users), then simply index that data in a separate path. Then you can have the complete data record in a single path and the names, emails, etc in a fast, query/sort-friendly list.

When is this useful?

This is an appropriate tool to use if you have a denormalized set of records that:

• cannot be merged practically into one path in a practical way
• have complex dependencies (A depends on C, and C depends on B, and B depends on A)
• records are written with high concurrency (i.e. possibly hundreds of write ops per minute to the SAME record by DIFFERENT users)

How do you do this?

The idea is to use update counters to ensure all paths stay at the same revision.

1) Create an update counter which is incremented using transactions:

function updateCounter(counterRef, next) {
   counterRef.transaction(function(current_value) {
      return (current_value||0)+1;
   }, function(err, committed, ss) {
      if( err ) console.error(err)
      else if( committed ) next(ss.val());
   }, false);
}

2) Give it some security rules

"counters": {
   "$counter": {
      ".read": true,
      ".write": "newData.isNumber() && ( (!data.exists() && newData.val() === 1) || newData.val() === data.val() + 1 )"
   }
},

3) Give your records security rules to enforce the update_counter

"$atomic_path": {
   ".read": true,
   // .validate allows these records to be deleted, use .write to prevent deletions
   ".validate": "newData.hasChildren(['update_counter', 'update_key']) && root.child('counters/'+newData.child('update_key').val()).val() === newData.child('update_counter').val()",
   "update_counter": {
      ".validate": "newData.isNumber()"
   },
   "update_key": {
      ".validate": "newData.isString()"
   }
}

4) Write the data with the update_counter

Since you have security rules in place, records can only successfully write if the counter does not move. If it does move, then the records have been overwritten by a concurrent change, so they no longer matter (they are no longer the latest and greatest).

var fb = new Firebase(URL);

updateCounter(function(newCounter) {
   var data = { foo: 'bar', update_counter: newCounter, update_key: 'myKey' };
   fb.child('pathA').set(data);
   fb.child('pathB').set(/* some other data */);
   // depending on your use case, you may want transactions here
   // to check data state before write, but they aren't strictly necessary
});

5) Rollbacks

Rollbacks are a bit more involved, but can be built off this principle:

• store the old values before calling set
• monitor each set op for failures
• set back to old values on any committed changes, but keep the new counter

A pre-built library

I wrote up a lib today that does this and stuffed it on GitHub. Feel free to use it, but please be sure you aren't making your life complicated by reading "Do you need this?" above.