efficiently display 100,000 items using Core Data

Question

I am using a NSFetchResultsController to display 100,000 + records in a UITableView. This works but it is SLOW, especially on an iPad 1. It can take 7 seconds to load which is torture for my users.

I'd also like to be able to use sections but this adds at least another 3 seconds onto the laod time.

Here is my NSFetchResultsController:

- (NSFetchedResultsController *)fetchedResultsController {

    if (self.clientsController != nil) {
        return self.clientsController;
    }

    NSFetchRequest *request = [[NSFetchRequest alloc] init];
    NSEntityDescription *entity = [NSEntityDescription entityForName:@"Client" inManagedObjectContext:self.managedObjectContext];
    [request setEntity:entity];
    [request setPredicate:[NSPredicate predicateWithFormat:@"ManufacturerID==%@", self.manufacturerID]];
    [request setFetchBatchSize:25];

    NSSortDescriptor *sort = [[NSSortDescriptor alloc]  initWithKey:@"UDF1" ascending:YES];
    NSSortDescriptor  *sort2= [[NSSortDescriptor alloc] initWithKey:@"Name" ascending:YES];
    [request setSortDescriptors:[NSArray arrayWithObjects:sort, sort2,nil]];

    NSArray *propertiesToFetch = [[NSArray alloc] initWithObjects:@"Name", @"ManufacturerID",@"CustomerNumber",@"City", @"StateProvince",@"PostalCode",@"UDF1",@"UDF2", nil];
    [request setPropertiesToFetch:propertiesToFetch];

    self.clientsController =
    [[NSFetchedResultsController alloc] initWithFetchRequest:request
                                        managedObjectContext:self.managedObjectContext sectionNameKeyPath:nil
                                                   cacheName:nil];

    return self.clientsController;

}

I have an index on ManufacturerID which is used in my NSPredicate. This seems like a pretty basic NSFetchRequest - anything I can do to speed this up? Or have I just hit a limitation? I must be missing something.

Answer 1

First: you can use the NSFetchedResultsController's cache to speed up display after the first fetch. This should quickly go down to a fraction of a second.

Second: you can try to display the only the first screenful and then fetch the rest in the background. I do this in the following way:

• When the view appears, check if you have the first page cache.
• If not, I fetch the first page. You can accomplish this by setting the fetch request's fetchLimit.
  • In case you are using sections, do two quick fetches to determine the first section headers and records.
• Populate a second fetched results controller with your long fetch in a background thread.
  • You can either create a child context and use performBlock: or
  • use dispatch_async().
• Assign the second FRC to the table view and call reloadData.

This worked quite well in one of my recent projects with > 200K records.