Realistic Data Backup method for Parse.com

Question

We are building an iOS app with Parse.com, but still can't figure out the right way to backup data efficiently.

As a premise, we have and will have a LOT of data store rows. Say we have a class with 1million rows, assume we have it backed up, then want to bring it back to Parse, after a hazardous situation (like data loss on production).

The few solutions we have considered are the following:

1) Use external server for backup

BackUp: - use the REST API to constantly back up data to a remote MySQL server (we chose MySQL for customized analytics purpose, since it's way faster and easier to handle data with MySQL for us)

ImportBack: a) - recreate JSON objects from MySQL backup and use the REST API to send back to Parse. Say we use the batch operation which permits 50 simultaneous objects to be created with 1 query, and assume it takes 1 sec for every query, 1million data sets will take 5.5hours to transfer to Parse.

b) - recreate one JSON file from MySQL backup and use the Dashboard to import data manually. We just tried with 700,000 records file with this method: it took about 2 hours for the loading indicator to stop and show the number of rows in the left pane, but now it never opens in the right pane (it says "operation time out") and it's over 6hours since the upload started.

So we can't rely on 1.b, and 1.a seems to take too long to recover from a disaster (if we have 10 million records, it'll be like 55 hours = 2.2 days).

Now we are thinking about the following:

2) Constantly replicate data to another app

Create the following in Parse: - Production App: A - Replication App: B So while A is in production, every single query will be duplicated to B (using background job constantly). The downside is of course that it'll eat up the burst limit of A as it'll simply double the amount of query. So not ideal thinking of scaling up.

What we want is something like AWS RDS which gives an option to automatically backup daily. I wonder how this could be difficult for Parse since it's based on AWS infra.

Please let me know if you have any idea on this, will be happy to share know-hows.

P.S.:

We’ve noticed an important flaw in the above 2) idea.

If we replicate using REST API, all the objectIds of all Classes will be changed, so every 1to1 or 1toMany relations will be broken.

So we think about putting a uuid for every object class.

Is there any problem about this method? One thing we want to achieve is query.include(“ObjectName”) ( or in Obj-C “includeKey”), but I suppose that won’t be possible if we don’t base our app logic on objectId.

Looking for a work around for this issue; but will uuid-based management be functional under Parse’s Datastore logic?

Answer 1

Parse has never lost production data. While we don't currently offer automated backups, you can request one any time you like, and we're working on making all of this even nicer. Additionally, it's easier in most cases to import the JSON export file through the data browser rather than using the REST batch.