Parse saveEventually failing when app is terminated quickly

Question

I have an app that uses parse to store data in parse's local data store and backs up said data to the parse cloud. Generally, this is working very well. The key bit of code to store data locally and in the cloud is as below :

- (void)store:(PFObject*) parseObject {
    if (parseObject) {
        [parseObject pinInBackground];
        [parseObject saveEventually];
    } else
        NSLog(@"Err :Store was passed a nil?");
}

I have an app where some users have said that there is data loss if they set their data and then terminate the app 'shortly' thereafter.

The function is passed around 10 items or more to store in quick succession when the user data is updated.

I have tested this scenario by doing the following. I let all the items be stored and set a breakpoint to be hit when this is done. I then let the app run again and terminate it by pressing the home key and swiping away the app. It has a further second or so of run time, but the key point is that the store has completed on each object in any case.

I do indeed find that data can be lost. It appears that just because these methods have run does not guarantee that the data will be stored. Just to be clear, I understand that these functions do not store the data, but I had thought (assumed) that the intent to store the data is guaranteed after they complete.

I would add the following :

1. Later data is more susceptible to loss. i.e. it appears that parse processes the data sequentially.
2. You can find the data is pinned in local datastore, but does not make it to parse (almost as though pin worked, but saveeventually did not).
3. Older (slower?) devices are much more susceptible than newer devices. In fact I struggle to make it happen on a new iPad mini, but can do so on iPhone 4.
4. Networking needs to be enabled, but it is easier to make it happen if you simulate a bad network (using iOS settings on the device).
5. Volume of data is small (100s bytes), I'm not hitting the parse save limit.
6. I am using Parse Version 1.8.4.

My question is as follows :

I was expecting that once these calls had returned the task was locked away and would always complete. I understand that there is no guarantee how long saveeventually may take, but that it would always complete 'eventually' even on a next run of the app. Have I done something wrong? Am I exposed to this kind data loss and need to take further precautions? Does anyone have any experience or suggestions? Even if it is simply that you find it works for you? Could be that I have done something silly elsewhere, but it is difficult to see how.

Thanks for your time.

Answer 1

I'm posting my own results here, maybe they will be useful for someone. It is my understanding of how it works after reviewing the Parse source, I can't guarantee that this is the final word or the whole story, but it fits the facts.

Parse is now open source so you can at least go and look to try to determine what is happening if you have a problem.

Parse in github

Parse keeps a queue of tasks, they are strictly processed in the order they are requested. This is true whether the task is requested in foreground (blocking) or in the background (with or without a callback). Also it doesn't matter if the task is a query, pin, save, etc., they are all queued up in the task list. The task list is kept in memory, if the app is suspended then terminated or just terminated before all the tasks are processed the task is lost.

So, if parse is not doing anything and you request a pin or a save, it will start this and in all likelihood all will be well.

If parse is busy (as per my description above) your saveEventually is on the end of the queue waiting to be processed. Until this task is processed you are at risk that the saveEventually is not logged by Parse. Note that it is important to understand that by process I mean looking at the task - not completing the task - just simply getting to look at what the task is and recording it in the case of saveEventually.

I don't necessarily think this is a bug. My problem was I was viewing the saveEventually as a database commit. I understand that this did not mean it was in the cloud yet, I just presumed the request to push it to the cloud was stored in a non-volatile manner when the saveEventually returned. Once it is processed it is like a commit, it will make it to the cloud eventually, but you cannot determine that it has been processed programmatically (the saveEventually callback is on completion not recording of the saveEventually).

If the app is nuked then so be it, the user has taken some action and they should know what they are doing. However, if you declare that your app has a background task, you can ensure that there is still a thread to execute if the user presses the home key or moves to another app - which I think they are entitled to do. I have done this and appear to prevent the data loss I was seeing. I am basically ensuring the task list is processed and Parse has noted all my saveEventuallys, even when the app goes to the background.

iOS Background Execution

It is hard to determine when to terminate this thread. In my app I do a pin and saveEventually in pairs. The Pin does have a callback so I keep an outstanding pin count. When it reaches zero, I wait an extra minute and shut down the thread. This gives 1 minute to execute the final saveEventually which seems more than adequate in testing. If I have access to the internet at that point, it doesn't matter the saveEventually is noted for later or the next run of the app.

I think you would have to be quite a heavy user of Parse to see this problem. I have lots of small data objects spread over different tables that are not related. This creates lots of queries and pins that get backed up, if you have one table or one data blob that you can pin in one go, you won't see the problem I describe.

Example Code (7Jan16 edit) as per comment request

Below is the core of the code that I use to achieve the above. You will need to change it if you use it. I recommend reading the background task link too above. The background thread keeps the app alive sufficiently that parse can still process its outstanding tasks.

Intention of this code block is to provide a helper function to keep track of all outstanding requests to parse. It starts a background thread (startPinMonitor) when it is called. Only one monitor is started, this is checked in startPinMonitor. My code runs on UI thread, but otherwise you might need some synchronisation logic here.

- (void)store:(PFObject*) parseObject {
    if (parseObject) {
        if (self.outStandingPins == 0)
            [self startPinMonitor];

        self.outStandingPins++;

        [parseObject pinInBackgroundWithBlock:^(BOOL succeeded, NSError *error) {
            self.outStandingPins--;
            if(!succeeded)
                NSLog(@"Err : Pin failed?");
        }];

        [parseObject saveEventually];
    }
}

This intention of this bit of code is to keep the app running until all remaining parse operations are dealt with. It first checks if the monitor is already active. The 'checkPinStatus' method is called every 30s and terminated when all pins have been processed. The log of outstanding pins is very useful to detect correct operation. You should be able to verify that the app is still running after pressing the home key. If you comment out the 'beginBackgroundTaskWithExpirationHandler' method you can determine the difference in behaviour that this code is trying to achieve.

-(void) startPinMonitor {
    if (![self.myTimer isValid]) {
        self.myTimer = [NSTimer scheduledTimerWithTimeInterval:30
                                                        target:self
                                  selector:@selector(checkPinStatus)
                                                      userInfo:nil
                                                       repeats:YES];

        self.myBackgroundTask = [[UIApplication sharedApplication]
                         beginBackgroundTaskWithExpirationHandler:^{
                             NSLog(@"Background tasks stopped");
                         }];
    }
}

-(void) checkPinStatus {
    NSLog(@"Current outStandingPins=%i", self.outStandingPins);

    if (self.outStandingPins == 0) {
        [self.myTimer invalidate];
        [[UIApplication sharedApplication]
            endBackgroundTask:self.myBackgroundTask];
    }
}