Best strategy for synching data in iPhone app

Question

I am working on a regular iPhone app which pulls data from a server (XML, JSON, etc...), and I'm wondering what is the best way to implement synching data. Criteria are speed (less network data exchange), robustness (data recovery in case update fails), offline access and flexibility (adaptable when the structure of the database changes slightly, like a new column). I know it varies from app to app, but can you guys share some of your strategy/experience?

For me, I'm thinking of something like this:

1) Store Last Modified Date in iPhone

2) Upon launching, send a message like getNewData.php?lastModifiedDate=...

3) Server will process and send back only modified data from last time.

4) This data is formatted as so:

<+><data id="..."></data></+> // add this to SQLite/CoreData

<-><data id="..."></data></-> // remove this

<%><data id="..."><attribute>newValue</attribute></data></%> // new modified value

I don't want to make <+>, <->, <%>... for each attribute as well, because it would be too complicated, so probably when receive a <%> field, I would just remove the data with the specified id and then add it again (assuming id here is not some automatically auto-incremented field).

5) Once everything is downloaded and updated, I will update the Last Modified Date field.

The main problem with this strategy is: If the network goes down when I am updating something => the Last Modified Date is not yet updated => next time I relaunch the app, I will have to go through the same thing again. Not to mention potential inconsistent data. If I use a temporary table for update and make the whole thing atomic, it would work, but then again, if the update is too long (lots of data change), the user has to wait a long time until new data is available. Should I use Last-Modified-Date for each of the data field and update data gradually?

Answer 1

I would start by making the update routine atomic, since you'll have enough on your hands figuring out how to get the client-server communication working properly.

After that is a good time to consider tweaking it to be incremental, but only after you do some testing to figure out if it's really necessary. If you're tuning your update protocol to be as low bandwidth as possible, you might discover that even a "big" update is downloaded fast enough.

Another way to look at it is to ask yourself, how often is there going to be network trouble when an average user is doing a sync? You probably don't want to tune for unlikely scenarios.

If you are trying to optimize (minimize) the data transfer you may want to consider a different format than XML, since XML is fairly verbose. Or at least you may want to trade in XML readability for space by making each element name and attribute as small as possible, and eliminate all unnecessary whitespace.