sync data from mongoDB to firebase and vice-versa

Question

My current situation:

I have created an application using React, NodeJS and Electron. Most of the users are a kind of offline users. They use my application offline.

Next plans:

Now, I am planning to create a mobile application for them. I plan to create that application using React-Native.

Since their database is offline, I planned to give them a sync to firebase button in desktop application. When he clicks on sync to firebase button, the data in their local mongodb should syncronize with firebase.

My thoughts:

• when a new record is added to mongodb, I will store a new key with that record which will look like: new: true.
• when a record is updated I will store a key named updated: true
• similarly for delete...

And then when user presses Sync to firebase, I will search for those records and add/update/delete respective records on firebase and then I will remove those keys from mongodb database.

Problems in executing my thoughts:

At first it does not smell me a good thing as I think it is time consuming because I will perform operations on firebase as well as mongodb.

Another problem with this approach is that if I think the other way round, that when user add/update/delete a record from React-Native app, firebase will have those keys line new/updated/deleted and then when user presses sync button in desktop application, I will have to do same thing but in reverse.

Yet another problem is that if user accidently uninstalled my application and then reinstalls it, then what should I do?

And the biggest problem is managing all the things.

My Expectations:

So, I want a clean and maintainable approach. Does any one have any idea on how to sync data from mongodb to firebase and vice-versa?

Answer 1

Both database systems supports for some sort of operation log or trigger system. You can use these to live update changes to databases to sync them almost real time.

For MongoDB

You can use Oplog to see what changes made to database (insert/update/delete) and run a suitable function to sync firebase.

oplog

A capped collection that stores an ordered history of logical writes to a MongoDB database. The oplog is the basic mechanism enabling replication in MongoDB.

There are small libraries that help you easily subscribe to these events.

Example (mongo-oplog)

import MongoOplog from 'mongo-oplog'
const oplog = MongoOplog('mongodb://127.0.0.1:27017/local', { ns: 'test.posts' })

oplog.tail();

oplog.on('op', data => {
  console.log(data);
});

oplog.on('insert', doc => {
  console.log(doc);
});

oplog.on('update', doc => {
  console.log(doc);
});

oplog.on('delete', doc => {
  console.log(doc.o._id);
});

For Firebase

You can use Cloud Functions. With Cloud Functions you can watch triggers like Cloud Firestore Triggers or Realtime Database Triggers and run a function to sync MongoDB database.

With Cloud Functions, you can handle events in the Firebase Realtime Database with no need to update client code. Cloud Functions lets you run database operations with full administrative privileges, and ensures that each change to the database is processed individually.

// Listens for new messages added to /messages/:pushId/original and creates an
// uppercase version of the message to /messages/:pushId/uppercase
exports.makeUppercase = functions.database.ref('/messages/{pushId}/original').onWrite((event) => {
  // Grab the current value of what was written to the Realtime Database.
  const original = event.data.val();
  console.log('Uppercasing', event.params.pushId, original);
  const uppercase = original.toUpperCase();
  // You must return a Promise when performing asynchronous tasks inside a Functions such as
  // writing to the Firebase Realtime Database.
  // Setting an "uppercase" sibling in the Realtime Database returns a Promise.
  return event.data.ref.parent.child('uppercase').set(uppercase);
});