What is the best approach to upload 1000+ records to a server that also contains images for each record from an iOS/Android app?

Question

I have a app working offline. It is assumed that 1000+ records are created with images in each record during this period and whenever connectivity is established. What should be the approach to send all the 1000+ records to server that also handles any interruption between the network calls or API failure response.

I assume I have to send records in batches but how to handle the interruption and maintain consistency and prevent any kind of data loss.

Answer 1

I guess the best way here is to send each record separetely (if they are not related to each other).

If you have media attachments, sending of each record will take 2 seconds in average, if you uploading via mobile internet with speed ~2 MB/s. If you will send the large batch of records via each request, you must have stable connection for a long period.

You can send each record as multipart request, where parts are record's body and media attachments.

Also you have no need to check for internet connection, or use receiver for catching changes of connection state. You can simply use this libraries for triggering sync requests:

1. JobScheduler
2. Firebase JobDispatcher
3. Evernote android-job

Answer 2

I would suggest to use Firebase database API. It has got nice offline/online/sync implementations.

https://firebase.google.com/docs/database/

And it is possible to read/write the data using Admin SDK for your NodeJS server:

https://firebase.google.com/docs/admin/setup

Answer 3

You can use divide and conquer approach means divide the task into small task and upload the data to the server. 1. take a boolean flag "isFinishData" starting with false. 2. starting upload the data on server from 0 to 100 records. 3. next record send from 100 to 200. 4. this process run until last record (1000) is not send . 5. in last record update set boolean variable true and exit from loop .

this logic would be work fine in IOS/android both.