online/offline data management

Question

I have to create an application that has functionality similar to the contacts app. You can add a contact on the client's iPhone and it should get uploaded onto the client's iPad. If the client updates the contact on their iPad, it should get updated on their iPhone.

Most of this is fairly straight forward. I am using Parse.com as my back end and saving contacts locally with Core Data. The only problem I'm encountering is managing contacts when the user is offline.

Let's say I have an iPhone and an iPad. Both of them currently have the same version of the online database. My iPhone is now offline. It is 9AM.

At 10AM I update the phone number for a contact on my iPad. It saves the change locally and online. At 11AM I update the email address for the same contact on my iPhone but I'm still offline.

At noon, my iPhone connects to the internet and checks the server for changes. It sees that its changes are more recent than the latest update (checking an updatedAt timestamp property), so instead of downloading the new phone number for the contact (which is "obsolete"), it overrides the phone number along with the email address (updates the new phone number to the old version it has because it was offline during the phone number update at 10AM and its changes are supposedly more recent).

How am I supposed to manage the online/offline problems encountered such as the one above? A solution I can think of would be to keep updated timestamps on every attribute for a contact instead of just a general updatedAt property for the entire contact, e.g. when was first name updated, when was last name updated, and then manually check if an offline device has more recent changes on every attribute instead of overwriting the whole object, but that seems sloppy.

I was also thinking on having an updatedLocally and updatedOnline timestamp property on every Core Data object. This way if the two don't match I can do a diff-check and use the most recent one for conflicts but this still doesn't seem like the cleanest solution. Has anyone else encountered something similar? If so, how did you solve it?

Pseudocode/Summary for what I think? covers every test case but still isn't very elegant/complete:

2 Entities on Parse.com: Contact and Contact History

Contact has first, last, phone, email, onlineUpdate

Contact History has a Primary Key to a Contact to refer to and the same attributes but with history. e.g. first: [{value:"josue",onlineUpdate:"9AM"},{value:"j",onlineUpdate:"10AM"},{value:"JOSUEESP",onlineUpdate:"11AM"}]

1 Entity on Core Data, Contact:

Contact has first, last phone, email, onlineUpdate, and offlineUpdate (IMPORTANT: this is only on Core Data, not on Parse)

for every contact in parse database as onlineContact {
    if onlineContact does not exist in core data {
        create contact in core data
    }
    else {
        // found matching local object to online object, check for changes
        var localContact = core data contact with same UID as onlineContact
        if localContact.offlineUpdate more recent than onlineContact.onlineUpdate {
            for every attribute in localContact as attribute {
                var lastOnlineValueReceived = Parse database Contact History at the time localContact.onlineUpdate for attribute
                if lastOnlineValueReceived == localContact.attribute {
                    // this attribute did not change in the offline update. use latest available online value
                    localContact.attribute = onlineContact.attribute
                }
                else{
                    // this attribute changed during the more recent offline update, update it online
                    onlineContact.attribute = localContact.attribute
                }
            }
        }
        else if onlineContact.onlineUpdate more recent than localContact.offlineUpdate {
            // another device updated the contact. use the online contact.
            localContact = offlineContact
        }
        else{
            // when a device is connected to the internet, and it saves a contact
            // the offline/online update times are the same
            // therefore contacts should be equivalent in this else statement
            // do nothing
        }
}

TL;DR: How are you supposed to structure a kind of version-control system for online/offline updates without accidental overwriting? I'd like to limit bandwidth usage to a minimum.

Answer 1

I would suggest to use key based updates instead of contact based updates.

You should not send the whole contact to the server, in most cases the user would just change a few attributes anyways (things like 'last name' usually don't change very often). This also reduces bandwith usage.
Along with the applied changes of your offline contact you send the old version number/last update timestamp of your local contact to the server. The server can now determine whether or not your local data is up to date, simply by looking at your old version number.
If your old version number matches the current version number of the server there is no need for your client to update any other information. If this is not the case the server should send you the new contact (after applying your requested update).

You can also save those commits, this would result in a contact history which does not store the whole contact each time a key was changed but only the changes themselves.

A simple implementation in pseudo code could look like this:

for( each currentContact in offlineContacts ) do
{

if( localChanges.length > 0){      // updates to be made
    commitAllChanges();
    answer = getServerAnswer();

    if(answer.containsContact() == true){  
                                  // server sent us a contact as answer so 
                                  // we should overwrite the contact
    currentContact = answer.contact;
    } else {
      // the server does not want us to overwrite the contact, so we are up to date!
    }
    // ... 

}
} // end of iterating over contacts

The server side would look just as simple:

for (currentContactToUpdate in contactsToUpdate) do 
{   
    sendBackContact = false;   // only send back the updated contact if the client missed updates
    for( each currentUpdate in incomingUpdates ) do {
        oldClientVersion = currentUpdate.oldversion;
        oldServerVersion = currentContact.getVersion();

       if( oldClientVersion != oldServerVersion ){
            sendBackContact = true;
            // the client missed some updates from other devices
            // because he tries to update an old version
       } 

       currentContactToUpdate.apply(currentUpdate);

    }

    if(sendBackContact == true){
       sendBack(currentUpdate);
    }
}

To get a better understanding of the workflow I will provide an example:

---

8 AM both clients and the server are up to date, each device is online

Each device has an entry (in this case a row) for the contact 'Foo Bar' which has the primary key ID. The version is the same for each entry, so all of them are up to date.

 _        Server    iPhone    iPad
 ID       42        42        42 
 Ver      1         1         1
 First    Foo       Foo       Foo
 Last     Bar       Bar       Bar
 Mail     f@b       f@b       f@b

(excuse this terrible format, SO sadly does not support any sort of tables...)

---

9 AM your iPhone is offline. You notice Foo Bar's email changed to 'foo@b'. You change the contact information on your phone like this:

UPDATE 42 FROM 1          TO 2             Mail=foo@b
 //    ^ID     ^old version  ^new version  ^changed attribute(s)

so now the contact in your phone would look like this:

 _        iPhone   
 ID       42       
 Ver      2       
 First    Foo      
 Last     Bar   
 Mail     foo@b   

---

10 AM your iPad is offline. You notice 'Foo Bar' is actually written as 'Voo Bar'! You apply the changes immediatly on your iPad.

UPDATE 42 FROM 1 TO 2 First=Voo

Notice that the iPad still thinks the current version of contact 42 is 1. Neither the server nor the iPad did notice how you changed the mail address and increased the version number, since no devices were connected to the network. Those changes are only locally stored and visible on your iPad.

---

11 AM you connect your iPad to the network. The iPad sends the recent update to the server.

Before:

 _        Server    iPad
 ID       42        42 
 Ver      1         2
 First    Foo       Voo
 Last     Bar       Bar
 Mail     f@b       f@b

iPad -> Server:

UPDATE 42 FROM 1 TO 2 First=Voo

The server can now see that you are updating Version 1 of contact 42. Since version 1 is the current version your client is up to date (no changes commited in the mean time while you were offline).

Server -> iPad

UPDATED 42 FROM 1 TO 2 - OK

After:

 _        Server    iPad
 ID       42        42 
 Ver      2         2
 First    Voo       Voo
 Last     Bar       Bar
 Mail     f@b       f@b

---

12 AM you disconnected your iPad from the network and connect your iPhone. The iPhone tries to commit the recent changes.

Before:

 _        Server    iPhone
 ID       42        42 
 Ver      2         2
 First    Voo       Voo
 Last     Bar       Bar
 Mail     f@b       foo@b

iPhone -> Server

UPDATE 42 FROM 1 TO 2 Mail=foo@b

The server notices how you try to update an old version of the same contact. He will apply your update since it is more recent than the iPad's update but will send you the new contact data to make sure you get the updated first name aswell.

After:

 _        Server    iPhone
 ID       42        42 
 Ver      2         2
 First    Voo       Voo
 Last     Bar       Bar
 Mail     foo@b     foo@b

Server -> iPad

UPDATED 42 FROM 1 TO 3 - Ver=2;First=Voo;.... // send the whole contact
/* Note how the version number was changed to 3, and not to 2, as requested.
*  If the new version number was (still) 2 the iPad would miss the update
*/

The next time your iPad connects to the network and has no changes to commit it should just send the current version of the contact and see whether it is still up to date.

---

Now you have committed two offline changes without overwriting each other.
You can easily extend this approach and so some optimizations.
For example:

• If the client tries to update an old version of the contact, don't send them the whole contact as answer. Rather send them the commits they missed and let them update their contact by themselves. This is useful if you store lots of information about your client and expect few changes to be done between updates.
• If the client updated all information about a contact we can assume he does not need to know about the missed updates, however we would let him know about everything he missed (but it would/should have no effect to him)

I hope this helps.