Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Migrate Firebase Realtime Database to Firestore

Tags:

I am looking for the best way to migrate my apps database which is using firebase realtime database to the new Cloud Firestore database. I am confident for the project I am working on I don't need to make any data schema changes, so I am pretty much just trying to 1-1 map it. Firebase has suggested on their site to just write a script to do this, but I am not sure of the best way to go about that. Has anyone already made a script that accomplishes this?

like image 454
Luke Avatar asked Oct 04 '17 19:10

Luke


People also ask

Can we use firestore and realtime database at the same project?

Using Cloud Firestore and Realtime Database You can use both databases within the same Firebase app or project. Both NoSQL databases can store the same types of data and the client libraries work in a similar manner. Keep in mind the differences outlined above if you decide to use both databases in your app.

Can I migrate data from Firebase?

You can do this with the Firebase CLI: firebase auth:export users. json --format=JSON --project your_project_id . Massage the exported user data into a format acceptable to your new provider, using whatever data transformation tools you are comfortable with. Upload the user data to the new provider.

Is firestore better than Firebase?

What are the differences between Firebase and Firestore? Firebase is a more compressive solution vs. Firestore and incorporates multiple services like databases, notifications, analytics, ML, etc. Firestore is a NoSQL database that is part of the Firebase app development platform.


2 Answers

I wrote up a little node script that migrated things in a quick and dirty way and it worked quite nicely.

It is below if anyone else is interested.

Note: This should only be used if your data model in the realtime database was completely flat and did not have much nested data, and you intend on keeping your data flat as well in Firestore

To run this script just create a node file called index.js and throw it in a directory along with your service account file and raw json file from the realtime database export and run the following from command line.

$ node index.js 

Script implementation below.

const admin = require('firebase-admin');  var serviceAccount = require("./config.json"); var database = require("./database.json"); var async = require ('async');  admin.initializeApp({   credential: admin.credential.cert(serviceAccount) });  var db = admin.firestore();  var allEntityNames = Object.keys(database);  var asyncTasks = [];  for (var i in allEntityNames) {   var entityName = allEntityNames[i];   var entity = database[entityName];   var entityKeys = Object.keys(entity);    console.log("began migrating "+ entityName);   for (var j in entityKeys) {     var entityKey = entityKeys[j];     var dict = entity[entityKey];     asyncTasks.push(function(callback){       db.collection(entityName).doc(entityKey).set(dict)         .then(function() {           callback();         })         .catch(function(error) {           console.log(error);           callback();         });     });   }   async.parallel(asyncTasks, function(){     console.log("Finished migrating "+ entityName);   }); } 
like image 176
Luke Avatar answered Oct 17 '22 03:10

Luke


Actually, I wrote a script in Node-Js that use batch in writing to Firestore (batch is super fast and suitable for write may items) here is my code, just change files name to your's name and run node YOUR_FILE_NAME.js

const admin = require('firebase-admin'); var serviceAccount = require('./firestore-config.json'); var database = require('./database.json');  admin.initializeApp({   credential: admin.credential.cert(serviceAccount),   databaseURL: 'YOUR_FILE_STORE_DB_URL', });  var db = admin.firestore(); var allEntityNames = Object.keys(database); var counter = 0; var commitCounter = 0; var batches = []; batches[commitCounter] = db.batch(); var ref = db.collection('users'); allEntityNames.forEach(function(k, i) {   if (counter <= 498) {     var thisRef = ref.doc(k);     batches[commitCounter].set(thisRef, database[k]);     counter = counter + 1;   } else {     counter = 0;     commitCounter = commitCounter + 1;     batches[commitCounter] = db.batch();   } }); for (var i = 0; i < batches.length; i++) {   batches[i].commit().then(function() {     console.count('wrote batch');   }); } 
  1. if you don't have Node-Js on your machine, google to install it. it is not so hard.
  2. you can download firestore-config.json from your firebase console.
like image 31
Ahmad Khani Avatar answered Oct 17 '22 05:10

Ahmad Khani