How can I improve core data fetch performance on the iPhone?

Question

Core data's performance on the iPhone is absolutely miserable. Is indexing completely broken or is it just a poor implementation?

I have about 21500 objects of a single type in my core data store (SQLite backing store). The objects are indexed on a UUID which is an NSString (for example, one looks like this: "6b09e200-07b6-11df-a245-002500a30d78").

A single fetch where the object exists using executeFetchRequest in NSManagedObjectContext takes about 0.75 seconds! This is with the simplest possible predicate of "uuid == $UUID" where $UUID is a string like the example above.

This is really surprising. If I wanted to fetch every object in my store, one by one, it would take nearly 4.5 hours!

Is there anyway to improve this performance or should I just abandon core data all together?

Answer 1

Several points. If it takes 5 seconds to fetch 21,500 rows, it sounds like you're running on an older device. Like a 3G or original iPhone. The memory and I/O performance on those is just plain slow. You'll need to handle your data with extreme care to avoid reading it all into memory and doing unnecessary I/O. You may find -setFetchBatchSize particularly useful. If you're running on a 3GS, 10-20 thousand rows is manageable but will require care. If you're on an ipad or iphone4, this shouldn't be much of an issue.

You don't need to create your own UUID, except to interface with an external system, like a server. Each Managed Object has an objectID which is an OOP representation of its primary key. Just pass the objectID around, and do queries like @"self = %@" or @"self IN %@" to search for an object by its ID or an array of ids. You can also use -existingObjectWithID:error: to look up just 1 object by its objectID which will be faster than a generic fetch request with a generic predicate.

The best way to verify the index is being used as you expect is to run the app in the simulator with the executable argument

-com.apple.CoreData.SQLDebug 1

that will log to console the SQL being generated. You should see some stuff ending in something like t0.uuid == ?

You can take that SQL select statement, and run it through SQLite's explain query facility. Run /usr/bin/sqlite3 against the db file in the simulator. Do

.explain ON explain query plan copythatsqllinehere

it should print out something like 0|0|TABLE ZFOO AS t0 WITH INDEX something

if it's missing "with index" then you have some issue with either the way you created the Core Data store (are you sure the model is marked to index uuid ?) or there's something else with your fetch request.

This is really surprising. If I wanted to fetch every object in my store, one by one, it >would take nearly 4.5 hours!

I suppose you could do it that way, as one of the most painful ways possible. Or you could use -setFetchBatchSize: and iterate over batches of the objects very quickly.

Also, keep in mind that each fetch does I/O with the database to remain in sync with what any other threads have saved. Fetching is not some magical dictionary look up. There's a lower bound on the time it takes to execute the smallest unit of I/O. You're going to want to amortize the number of individual I/O requests to get the best performance. You'll have to balance that against reading too much into memory at once.

If you continue to have trouble, please file a bug with bugreport.apple.com

• Ben

Answer 2

This isn't going to answer your question but might give you something to think about. Using just SQLite on the iPhone I was mightily disappointed with performance. I was dealing with about 8000 entries which would take about a two minutes to insert / sort if returning all and so on.

Playing around with it I found the time needed to filter / sort in memory was 100 fold better than letting it be done SQLite and I think its mainly due to the performance of the flash memory.

In short the less Core Data has to use the flash memory the better performance you will get and I don't think there would be many ways to make it much better.