I've read the Firebase docs on Stucturing Data. Data storage is cheap, but the user's time is not. We should optimize for get operations, and write in multiple places.
So then I might store a list node and a list-index node, with some duplicated data between the two, at very least the list name.
I'm using ES6 and promises in my javascript app to handle the async flow, mainly of fetching a ref key from firebase after the first data push.
let addIndexPromise = new Promise( (resolve, reject) => { let newRef = ref.child('list-index').push(newItem); resolve( newRef.key()); // ignore reject() for brevity }); addIndexPromise.then( key => { ref.child('list').child(key).set(newItem); });
How do I make sure the data stays in sync in all places, knowing my app runs only on the client?
For sanity check, I set a setTimeout in my promise and shut my browser before it resolved, and indeed my database was no longer consistent, with an extra index saved without a corresponding list.
Any advice?
Denormalizing a database requires data has first been normalized. With denormalization, the database administrator selectively adds back specific instances of redundant data after the data structure has been normalized. A denormalized database should not be confused with a database that has never been normalized.
A denormalized database should never be confused by a database that has never been normalized. Example: Suppose after normalization we have two tables first, Student table and second, Branch table. The student has the attributes as Roll_no, Student-name, Age, and Branch_id.
Denormalization is a database optimization technique in which we add redundant data to one or more tables. This can help us avoid costly joins in a relational database.
Data Denormalization is a technique used on a previously-normalized database to increase the performance. In computing, denormalization is the process of improving the read performance of a database, at the expense of losing some write performance, by adding redundant copies of data or by grouping it.
Great question. I know of three approaches to this, which I'll list below.
I'll take a slightly different example for this, mostly because it allows me to use more concrete terms in the explanation.
Say we have a chat application, where we store two entities: messages and users. In the screen where we show the messages, we also show the name of the user. So to minimize the number of reads, we store the name of the user with each chat message too.
users so:209103 name: "Frank van Puffelen" location: "San Francisco, CA" questionCount: 12 so:3648524 name: "legolandbridge" location: "London, Prague, Barcelona" questionCount: 4 messages -Jabhsay3487 message: "How to write denormalized data in Firebase" user: so:3648524 username: "legolandbridge" -Jabhsay3591 message: "Great question." user: so:209103 username: "Frank van Puffelen" -Jabhsay3595 message: "I know of three approaches, which I'll list below." user: so:209103 username: "Frank van Puffelen"
So we store the primary copy of the user's profile in the users
node. In the message we store the uid
(so:209103 and so:3648524) so that we can look up the user. But we also store the user's name in the messages, so that we don't have to look this up for each user when we want to display a list of messages.
So now what happens when I go to the Profile page on the chat service and change my name from "Frank van Puffelen" to just "puf".
Performing a transactional update is the one that probably pops to mind of most developers initially. We always want the username
in messages to match the name
in the corresponding profile.
Using multipath writes (added on 20150925)
Since Firebase 2.3 (for JavaScript) and 2.4 (for Android and iOS), you can achieve atomic updates quite easily by using a single multi-path update:
function renameUser(ref, uid, name) { var updates = {}; // all paths to be updated and their new values updates['users/'+uid+'/name'] = name; var query = ref.child('messages').orderByChild('user').equalTo(uid); query.once('value', function(snapshot) { snapshot.forEach(function(messageSnapshot) { updates['messages/'+messageSnapshot.key()+'/username'] = name; }) ref.update(updates); }); }
This will send a single update command to Firebase that updates the user's name in their profile and in each message.
Previous atomic approach
So when the user change's the name
in their profile:
var ref = new Firebase('https://mychat.firebaseio.com/'); var uid = "so:209103"; var nameInProfileRef = ref.child('users').child(uid).child('name'); nameInProfileRef.transaction(function(currentName) { return "puf"; }, function(error, committed, snapshot) { if (error) { console.log('Transaction failed abnormally!', error); } else if (!committed) { console.log('Transaction aborted by our code.'); } else { console.log('Name updated in profile, now update it in the messages'); var query = ref.child('messages').orderByChild('user').equalTo(uid); query.on('child_added', function(messageSnapshot) { messageSnapshot.ref().update({ username: "puf" }); }); } console.log("Wilma's data: ", snapshot.val()); }, false /* don't apply the change locally */);
Pretty involved and the astute reader will notice that I cheat in the handling of the messages. First cheat is that I never call off
for the listener, but I also don't use a transaction.
If we want to securely do this type of operation from the client, we'd need:
username
fields for messages by so:209103
to null
(some magic value)name
of user so:209103
to 'puf'username
in every message by so:209103
that is null
to puf
.and
of two conditions, which Firebase queries don't support. So we'll end up with an extra property uid_plus_name
(with value so:209103_puf
) that we can query on.This type of approach makes my head hurt. And usually that means that I'm doing something wrong. But even if it's the right approach, with a head that hurts I'm way more likely to make coding mistakes. So I prefer to look for a simpler solution.
Update (20150925): Firebase released a feature to allow atomic writes to multiple paths. This works similar to approach below, but with a single command. See the updated section above to read how this works.
The second approach depends on splitting the user action ("I want to change my name to 'puf'") from the implications of that action ("We need to update the name in profile so:209103 and in every message that has user = so:209103
).
I'd handle the rename in a script that we run on a server. The main method would be something like this:
function renameUser(ref, uid, name) { ref.child('users').child(uid).update({ name: name }); var query = ref.child('messages').orderByChild('user').equalTo(uid); query.once('value', function(snapshot) { snapshot.forEach(function(messageSnapshot) { messageSnapshot.update({ username: name }); }) }); }
Once again I take a few shortcuts here, such as using once('value'
(which is in general a bad idea for optimal performance with Firebase). But overall the approach is simpler, at the cost of not having all data completely updated at the same time. But eventually the messages will all be updated to match the new value.
The third approach is the simplest of all: in many cases you don't really have to update the duplicated data at all. In the example we've used here, you could say that each message recorded the name as I used it at that time. I didn't change my name until just now, so it makes sense that older messages show the name I used at that time. This applies in many cases where the secondary data is transactional in nature. It doesn't apply everywhere of course, but where it applies "not caring" is the simplest approach of all.
While the above are just broad descriptions of how you could solve this problem and they are definitely not complete, I find that each time I need to fan out duplicate data it comes back to one of these basic approaches.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With