Core Data sqlite-wal file gets MASSIVE (>7GB) when inserting ~5000 rows

Question

I'm importing data into Core Data and find that the save operation is slow. Using the iOS simulator, I watch the sqlite-wal file grow and grow until its over 7GB in size.

I'm importing approx 5000 records with about 10 fields. This isn't a lot of data.

Each object I'm inserting has a to-one relation to various other objects (6 relations total). All of those records combined equal less than 20 fields. There are no images or any binary data or anything that I can see that would justify why the resulting size of the WAL file is so huge.

I read the sqlite docs describing the wal file and I don't see how this can happen. The source data isn't more than 50 MB.

My app is multi-threaded. I create a managed object context in the background thread that performs the import (creates and saves the core data objects).

Without writing the code out here, has anyone encountered this? Anyone have a thought on what I should be checking. The code isn't super simple and all the parts would take time to input here so lets start with general ideas.

I'll credit anyone who gets me going in the right direction.

Extra info:

• I've disabled the undo manager for the context as I don't need that (I think it's nil by default on iOS but I explicitly set it to nil).
• I only call save after the entire loop is complete and all managed objects are in ram (ram goes up to 100 MB btw).
• The loop and creation of the core data objects takes only 5 seconds or so. The save takes almost 3 minutes as it writes the the awl file.

Answer 1

It seems my comment to try using the old rollback(DELETE) journal mode rather than WAL journal mode fixed the problem. NOTE that there seem to be a range of problems when using WAL journal mode including the following:

• this problem
• problems with database migrations when using the migratePersistentStore API
• problems with lightweight migrations

Perhaps we should start a Core Data WAL problems page and get a comprehensive list and ask Apple to fix the bugs.

Note that the default mode under OS X 10.9 and iOS 7 now uses WAL mode. To change this back add the following option

@{ NSSQLitePragmaOptions : @{ @"journal_mode" : @"DELETE" } }