I'm now stuck for about two weeks with a nasty Core Data problem. I read lots of blogpost, articles and SO questions/answers but I'm still not able to solve my problem.
I ran lots of tests and was able to reduce the larger problem to a smaller one. It's going to be a large explanation so keep with me!
I have to got following datamodel:
Object A has one-to-many relation with object B which has another one-to-many relation with object C. Because of Core Data recommendations I have to create inverse relations so each instance of B points to its parent A and the same for C which points to its parent B.
A <->> B <->> C
To keep responsiveness smooth as butter I created a three-level managedObjectContext structure.
NSPrivateQueueConcurrencyType
, is tight to the persistentStoreCoordinator
NSMainQueueConcurrencyType
and has parent MOC 1My main datacontroller is added as an observer to the NSManagedObjectContextDidSave
notification of MOC 2 so every time MOC 2 saves a performBlock:
on MOC1 is triggered which performs a save operation (asynchronously because of performBlock:
).
To perform parsing a large JSON file into my Core Data structure I wrote a recurrent parser. This parser starts by creating a new MOC (3). It then takes the data for object A and parses its properties. Then the parser reads out the JSON relations for B and create the corresponding objects which are filled with data. These new objects are added to A by calling addBObject:
on A.
Because the parser is recurrent, parsing B means parsing C and here also new objects are created and attached to B.
This all happens in the performBlock:
on MOC 3.
After each parsing operation I save MOC 3 and dispatches on the mainThread a save operation of the main MOC (2). Because of the NSManagedObjectContextDidSave
notification MOC 1 will autosave asynchronously.
if (parsed){
NSError *error = nil;
if (![managedObjectContext save:&error])
NSLog(@"Error while saving parsed data: %@", error);
}else{
// something went wrong, discard changes
[managedObjectContext reset];
}
dispatch_async(dispatch_get_main_queue(), ^{
// save mainQueueManagedObjectContext
[[HWOverallDataController sharedOverallDataController] saveMainThreadManagedObjectContext];
});
To release my memory footprint and because I do not need to parsed data for now I am performing:
[a.managedObjectContext refreshObject:a mergeChanges:NO];
for each A I just parsed.
Because I need to parse about 10 A's which all have about 10 B's which have all about 10 C's a lot of managedObject's are generated.
Everything works fine. The only thing is: when I turn on the Allocations tool I see unreleased A's, B's and C's. I don't get any useful information from their retainCounts or whatsoever.
And because my actual problem regards a more complex dataModel the living objects become a serious memory problem.
Can someone figure out what I'm doing wrong? Calling refreshObjects on the other managedObjectContexts with the correct managedObject does not work either. Only a hard reset
seems to work but then I loose my pointers to living objects used by the UI.
I tried creating unidirectional relations instead of bidirectional ones. This create a lot other problems which cause Core Data inconsistencies and weird behavior (such as dangling objects and Core Data generating 1-n relations instead of n-n relations (because the inverse relation is not known).
I tried refreshing each changed or inserted object when I retrieve a NSManagedObjectContextDidSave
notification on any object
These both 'solutions' (which don't work by the way) seems also a bit hacky. This should not be the way to go. There should be a way of getting this to work without raising the memory footprint and by keeping the UI smooth, though?
http://cl.ly/133p073h2I0j
After refreshing every object ever used (which is tedious work) in the mainContext (after a mainSave) the object their sizes are reduced to 48 bytes. This indicates that the objects are all faulted, but that there is still a pointer left in memory. When we have about 40.000 objects which are all faulted there is still 1.920 MB in memory which is never released until the persistentManagedObjectContext is reset. And this is something we don't want to do because we loose every reference to any managedObject.
Robin,
I have a similar problem which I solved differently than you have. In your case, you have a third, IMO, redundant MOC, the parent MOC. In my case, I let the two MOCs communicate, in an old school fashion, through the persistent store coordinator via the DidSave notifications. The new block oriented APIs make this much simpler and robust. This lets me reset the child MOCs. While you gain a performance advantage from your third MOC, it isn't that great of an advantage over the SQLite row cache which I exploit. Your path consumes more memory. Finally, I can, by tracking the DidSave notifications, trim items as they are created.
BTW, you are also probably suffering from a massive increase in the size of your MALLOC_TINY
and MALLOC_SMALL
VM regions. My trailing trimming algorithm lets the allocators reuse space sooner and, hence, retards the growth of these problematic regions. These regions are, in my experience, due to their large resident memory footprint a major cause for my app, Retweever, being killed. I suspect your app suffers the same fate.
When the memory warnings come, I call the below snippet:
[self.backgroundMOC performBlock: ^{ [self.backgroundMOC reset]; }];
[self.moc save];
[self.moc.registeredObjects trimObjects];
-[NSArray(DDGArray) trimObjects]
just goes through an array and refreshes the object, thus trimming them.
In summary, Core Data appears to implement a copy on write algorithm for items that appear in many MOCs. Hence, you have things retained in unexpected ways. I focus upon breaking these connections after import to minimize my memory footprint. My system, due to the SQLite row cache, appears to performa acceptably well.
Andrew
For every NSManagedObjectContext
that you keep around for a specific purpose you are going to accumulate instances of NSManagedObject
A NSManagedObjectContext
is just a piece of scratch note paper that you can instantiate at will and save if you wish to keep changes in the NSPersistentStore
and then discard afterward.
For the parsing operations (layer 3) try creating a MOC for the op , do your parsing, save the MOC and then discard it afterwards.
It feels like you have at least one layer of MOC being held in strong references too many.
Basically ask the question for each of the MOC's. "Why am keeping this object and its associated children alive".
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With