Merge conflict when multiple core data inserts are dispatched to NSPersistentContainer using performBackgroundTask

Question

I was experimenting with the new core data API NSPersistentContainer and was under the impression that an internal queueing mechanism would prevent write transactions from evaluating concurrently, as detailed in this stack overflow answer NSPersistentContainer concurrency for saving to core data

The way that a lot of pros have been dealing with the problem for a long time (even before NSPersistentContainer did it) was to have an operation queue to queue the writes so there is only one write going on at a time, and have another context on the main thread only for reads. This way you never get any merge conflicts. (see https://vimeo.com/89370886 for a great explanation on this setup which is now what NSPersistentContainer does internally). When you call performBackgroundTask the persistentContainer enqueues that block into an internal serial queue. This ensure that there are no mergeConflicts.

However if I insert multiple entities which share a relationship destination in a tight loop using performBackgroundTask for each iteration, I get a NSMergeConflict error when I save the context:

            let bundlePath = Bundle.main.resourceURL!

        let directoryEnumerator = FileManager.default.enumerator(at: bundlePath, includingPropertiesForKeys: [URLResourceKey.isDirectoryKey, URLResourceKey.nameKey])
        while let url = directoryEnumerator?.nextObject() as? URL {

            if url.pathExtension == "jpeg" {
                let imageData = try! Data(contentsOf: url)

                DataManager.persistentContainer.performBackgroundTask { (context) in

                    //          context.mergePolicy = NSMergePolicy.overwrite

                    let new = Photo(context: context)
                    new.name = url.lastPathComponent
                    new.data = imageData as NSData

                    let corresponding = try! context.existingObject(with: DataManager.rootFolder.objectID) as! Folder
                    new.parent = corresponding

                    try! context.save()
                }
            }

I posted a sample project on github to demonstrate the problem: https://github.com/MaximeBoulat/NSPersistentContainer_Merge_Conflict

The crash seems to be happening because multiple entities are setting their "parent" relationship concurrently, for the same parent, which causes the parent's "children" relationship to be desynchronized across concurrent updates.

This happens even if I set the .automaticallyMergesChangesFromParent property of the incoming context to true. I can prevent the crash by defining the incoming context's merge policy, but that's not an acceptable solution.

Is there any way to configure NSPersistentContainer to properly serialize updates dispatched using the performBackgroundTask API. Or is there something I am missing which is causing these updates to conflict with each other? Or did Apple provide the NSPersistentContainer stack with the expectations that any conflicts encountered when evaluating logic passed into performBackgroundTask should either be fatal, or be disregarded?

Answer 1

I wrote the answer that you are quoting. I was wrong. I have updated it.

I have found that NSPersistentContainer's performBackgroundTask does not have a functional internal queue and it can lead to merge conflicts. When I initially tested it, it seemed like it did, but I found out like you that there can be conflicts. Luckily it is not that hard to fix by creating your own queue. I know it seems strange for Apple to release something that is so broken, but that seems to be the case.

I am sorry for posted incorrect information.

Answer 2

From the documentation on performBackgroundTask(:):

Each time this method is invoked, the persistent container creates a new NSManaged​Object​Context with the concurrency​Type set to private​Queue​Concurrency​Type. The persistent container then executes the passed in block against that newly created context on the context’s private queue

So, I don't think that's doing what you want it to do. I think you want to call newBackgroundContext(), store it in a property somewhere, and use performBlock(:) on it whenever you want that serialization.

Answer 3

There’s a bug in your code, line 77 of your DataManager is on a background thread and calls rootFolder which then uses the viewContext. You should not use the viewContext on a background thread. You need to already have the objectID before the background thread begins, you could move the rootFolder.objectID above the block.