Core Data Saves and UI Performance

Question

I'm wondering if there are any best practices for improving UI responsiveness while doing Core Data saves (not fetches) with large collections of managed objects.

The app I'm working on needs to download fairly large amounts of data on set intervals from a web service until complete. On each interval, a batch of data is downloaded, formatted into managed objects, and saved to Core Data. Because this process can sometimes take as long as 5 minutes until fully complete, simply adding a loading screen until everything finishes is not really an option, it takes too long. I'm also interested in doing frequent writes to Core Data, rather than one big write at the end, to keep my memory footprint low. Ideally, I'd like the user to be able to keep using the rest of the application normally, while simultaneously downloading and writing these large data sets to Core Data.

Unfortunately, what seems to be happening is that when I try to save my inserts that I put into the managed object context for each batch, that save operation blocks the user from interacting with the rest of the app (swiping tables, touching buttons, etc) until complete. For those short periods of time where a Core Data save is taking place, the app is very unresponsive.

Naturally, I've tried making those saves smaller by reducing the size of the individual batches that get downloaded per interval, but besides the inconvenience of making the whole process take longer, there will still be instances when a user's swipe is not captured, because at that particular time a core data save was happening. Reducing the size simply makes it less likely that a missed swipe or a missed touch will happen, but they still seem to happen often enough to be inconvenient.

For the inserts themselves, I've tried using two different implementations: insertNewObjectForEntityForName:inManagedObjectContext as well as setValuesForKeysWithDictionary. Both exhibit the problem I described above.

I tried prototyping a much simpler test to see performance in both the simulator and on the device, I've provided the important elements here. This example doesn't actually download anything from the web, but just writes a whole bunch of stuff to core data on set intervals from within a TableViewController. I'd love to know if anyone has any suggestions to improve responsiveness.

- (void)viewDidAppear:(BOOL)animated 
{
    [super viewDidAppear:animated];
    timer = [NSTimer scheduledTimerWithTimeInterval:1 target:self selector:@selector(doTimerWork:) userInfo:nil repeats:YES];
}

-(void) doTimerWork:(id)sender
{
    for (int i = 0; i < 1000; i++)
    {
        Misc * m = (Misc*)[NSEntityDescription insertNewObjectForEntityForName:@"Misc" inManagedObjectContext:managedObjectContext];        
        m.someDate = [NSDate date];
        m.someString = @"ASDASDASD";
        m.someOtherString = @"BLAH BLAH BLAH";
        m.someNumber = [NSNumber numberWithInt:5];
        m.someOtherNumber = [NSNumber numberWithInt:99];
        m.someOtherDate = [NSDate date];    
    }

    NSError *error;
    if (![managedObjectContext save:&error]) {
        NSLog(@"Experienced an error while saving to CoreData");
    }
}

Answer 1

Typically you would download your data on a background thread and insert/update managed objects into its managed object context.
On the main thread you would register and receive the NSManagedObjectContextWillSaveNotification and use mergeChangesFromContextDidSaveNotification: to update the main managed object context.

Is this what you are doing?

Also, read Multi Threading with Core-Data.