Optimistic locking support in NSIncrementalStore subclass

Question

I am implementing a custom NSIncrementalStore subclass which uses a relational database for persistent storage. One of the things that I still struggle with is the support for optimistic locking.

---

(feel free to skip this lengthy description right to my question below)

I analyzed how Core Data's SQLite incremental store approaches this problem by examining SQL logs produced by it and came up with following conclusions:

• Each entity table in the database has a Z_OPT column which indicates the number of times a particular instance of this entity (row) has been modified, starting from 1 (initial insertion).

• Each time a managed object is modified, Z_OPT value in its corresponding database row is incremented.

• The store maintains cache (referred to as row cache in Core Data docs) of NSIncrementalStoreNode instances, each having a version property equal to Z_OPT value returned by previous SELECT or UPDATE SQL query on managed object's row.

• When a managed object is returned from NSManagedObjectContext (e.g. by executing NSFetchRequest on it), MOC creates snapshot of this object which contains this version number.

• When the object is modified or deleted, Core Data makes sure that it has not been modified or deleted outside the context by comparing versions of cached row and object snapshot. All of this happen when -save: is called on the context that the object belongs to. If the versions are different then a merge conflict is detected and handled based on set merging policy.

When MOC is being saved, the -newValuesForObjectWithID:withContext:error: method is called for each modified/deleted object which in turn returns NSIncrementalStoreNode with version number. This version is then compared to snapshot's version and if they are different, the save fails with appropriate merge conflicts (at least with default merge policy).

This simple use case works properly with my store since -newValuesForObjectWithID:withContext:error: checks the row cache first which is enough if the object was concurrently modified in other context using the same store instance. If this is the case, then the cache contains updated row with higher version number which is enough to detect a conflict.

But how can I detect than the underlying database has been modified outside my store, possibly by other application or other store instance using the same database file? I know this is an unfrequent edge case but Core Data handles it properly and I would prefer to do the same.

Core Data's store uses SQL queries like these to update/delete object's row:

UPDATE ZFOO SET Z_OPT=Y, (...) WHERE (...) AND Z_OPT=X
DELETE FROM ZFOO WHERE (...) AND Z_OPT=X

where:
X - version number last known to the store (from cache)
Y - new version number

If such a query fails (no rows affected) the row is updated in store's cache and its version compared against the one previously cached.

---

My question is: how can a custom NSIncrementalStore inform Core Data that optimistic locking failure has occurred for some updated/deleted/locked objects? It is only the store that is able to tell that when it handles NSSaveChangesRequest passed to it its -executeRequest:withContext:error: method.

If the underlying database does not change under the store, then conflicts are detected since Core Data calls -newValuesForObjectWithID:withContext:error: on each modified/deleted/locked object prior to executing save changes request on the store. I was not able to find any way for NSIncrementalStore to inform Core Data that an optimistic locking failure has occurred after it started to handle the save request. Is there some undocumented way to do that? Core Data seems to throw some exception in that case which is then magically translated into failed save request with NSError listing all the conflicts. I am only able to mimic that partly by returning nil from -executeRequest:withContext:error: and creating the error message by myself. I think there must be a way to use the standard Core Data conflict handling mechanism in this scenario as well.

Answer 1

I realize that this is not an answer to you question, but I will try and give you my point of view on CoreData and correlation to Databases:

(1st level cache)
NSPesistentStoreCoordinator + NSPersistentStore == A single connection to the database

(2nd level cache)
NSManagedObjectContext == cache over the connection holding changes

So, to my understanding your issue is that you have multiple connections to your store, each making changes, but you have no central version control over your records. Your store will receive a -executeRequest:withContext:error: with NSSaveRequestType
You will then be responsible to verify that the record versions match, if you find a conflict in the connection level (level 1) you report version mismatch between the context (level 2) and the coordinator.
you need to report version missmatch between your connection (level 1) and your store.
To be able to do this your store must report changes on it across all connections to it (ConnectionManager), or it might offer hooks to changes performed on it.
I'm no SQLite expert, but the SQLite API does have something to offer in that area:
update hook
commit hook
changes
total changes
(I have no experience in setting these kind of hooks, but if CoreData use them it will not show in the debug logs)

you can report these errors by setting the error pointer (NSError**) and setting its internal data to match the one that CoreData coordinator is setting (create merge conflict and set the information in them as needed)

Note that optimistic locking failure will only occur during -executeRequest:withContext:error: (unless you have a rogue connection to the store, one that is not tracked by the manager.
To support this behaviour your manager might need to verify each record as it is committed for a save [huge performance cost] , or use some hooks into the changes recently made to records )

To handle multiple connections to your store you might need to have a shared cache of NSIncrementalStoreNode, keyed by the store url:
static @{
url1 : actualCacheMapping1,
url2 : actualCacheMapping2,
...
}
each connection save to the store will be verified agains the store url actual cache.

Hope this make some sense for you.