iPhone memory warnings and crashes - but Instruments showing lowish memory use

Question

I have a strange memory issue I'm having problems resolving and would appreciate some advice as to where else to look.

The program I have (iPhone App) has a function whereby it basically downloads loads of files, processes those that are JSON, and stores the rest to disk. The JSON processing is CPU intensive and can take several seconds per file, so I have a NSOperationQueue with maxConcurrency limited to 1 that handles all the heavy lifting, and a queue that manages the multiple files to download.

Ever since iOS5 came out, the App has had problems completing the download sequence without crashing and so far what I have tried is;

1) Changed the performSelectorOnBackgroundThread JSON processing to use a single NSOperationQueue so as to limit the number of background threads working with large objects.

2) Added NSAutoReleasePools inside loops that create multiple, large, transient objects.

3) Flushed the sharedURLCache to ensure the files aren't hanging around in the system cache.

4) Stored the JSON objects to disk using NSKeyedArchiver and passed the filenames between threads rather than the actual objects, to again try to mitigate the number and size of retained objects currently in use.

All of these at first seemed to make a difference, and when I look at the memory allocations, I've now got the peak usage down from just over 20MB (hence no wonder it was crashing) to under 10MB, and yet the app is still crashing with low memory as before.

I'm trying to trace what is eating the memory causing the app to crash and on this occasion I'm having real problems persuading Instruments to tell me anything useful.

Here's a typical trace (on an iPhone 3GS running iOS 4.3.5)

You can see that the PEAK usage was a tad over 7MB and yet shortly after, you can see the 2 flags pertaining to low memory, and then low memory urgent, followed by the app terminating shortly thereafter.

If I use the memory monitor, the cause of the crash seems clear enough - physical memory is being exhausted - look at the light green trace below. The low memory warnings co-incide (not surprisingly) with the physical memory running out.

There are no leaks showing FWIW either (I've done that in other runs).

It's not image caches or NSURLConnection caches and the only thing I can think of is that perhaps there are some bad leaks that aren't being detected ... but I'm having issues identifying them because if I click into all allocations to see the objects that are live, and then do a command-A to select them all (in order to paste them into a spreadsheet to see where the memory seems to be), at the point I hit command-C to copy them, Instruments beachballs and never recovers.

I really cant figure out what's going on. Does anyone have some advice on how to persuade instruments to show me some more useful information about what is using this memory?

Sorry I can't post any meaningful code fragments ... hopefully the instruments screenshots at least give you an idea about where I'm coming from.

Answer 1

The Leaks instrument isn't terribly useful for figuring out anything but the obvious leaks in your app.

What you are describing is an ideal candidate for heapshot analysis.

tl;dr Heapshot analysis allows you to see exactly how the heap of your application grows between any two points of time (where you determine the points).