Where is my AVPlayer's memory, and how do I get it back?

Question

I'm playing heaps of videos at the same time with AVPlayer. To reduce loading times, I'm storing the corresponding views in a NSCache.

This works fine until reaching a certain number of videos, from which the videos simply stop playing, or even appearing.

There's no error, log or memory warning. In particular, I'm listening to UIApplicationDidReceiveMemoryWarningNotification to clear the cache but this is never received.

If I remove the cache, all the videos play at expense of worse performance.

This makes me suspect that AVPlayer is using memory from a different process (which one?). And when that memory reaches a certain limit, new players cease to work.

Is this correct?

If so, is there a way to be notified when this magic limit is reached to take the appropriate measures (e.g., clear the cache) to ensure playback of other media?

Answer 1

Good news and bad news - good is you can probably fix the problem, bad is it takes work and is somewhat complex.

Root Problem

The reason you don't get notified early happens because iOS does not find out that your app has exceeded its memory budget til its almost too late, then it immediately kills it. The problem has to do with the way iOS (and OS X) manage the file system cache. Normally, when files get opened, as you read the data, the file data gets transferred into a buffer in the Unified Buffer Cache (a term you can google for more info) - I'll call it UBC from now on.

So suppose you have 10 open files, and you have read every file to the end, but have not closed the files. Well, all that data is sitting in the UBC. Now, if you close the files, the buffers are all freed. And technically, the OS can purge this buffers too - only it seems by the time it realizes that memory is tight, it chooses to blow the app away first (and there may be valid reasons for it to do this). So imagine that you app is showing videos, and the way the videos get loaded is through the file system, the number of free buffers starts dropping. At some point iOS notices this, tracks down who most belong to (your app), and wham, kills your app ASAP.

I hit this problem myself in an open source project I support, PhotoScrollerNetwork. Users started complaining that their project was getting terminated by the system, like you, without any notification. I tried in vain to monitor the UBC (there are APIs on OS X to do so, but not on iOS). In the end I found a solution using an heuristic - monitor all your memory usage including the UBC, and don't exceed 50% of the total available iOS memory pool.

So (you might ask) - what is the Apple approved way to solve this problem? Well, there is none. How do I know that? Because I had a half hour long discussion at WWDC 2012 with the Director of Core iOS in one of the labs (after getting ping ponged around by others who had no idea what I was talking about). In the end, after I explained the above heuristic, he told me directly that the solution was probably as good as any he could think of. Without an API to directly monitor the UBC, you can only approximate its usage and adjust accordingly.

But you say, I'm using the NSCache - why doesn't the system account for the AVPlayer memory there? There reason is undoubtedly the UBC - an AVPlayer instance probably only consumes a few thousand K of memory itself - its the open file to the video that is not accounted for by iOS.

Possible Solutions

1) If you can load the videos directly into a NSData object, and keep that in the NSCache, you can most likely totally avoid the UBC issues mentioned above. [I don't know enough about the AV system to know if you can do this.] In this case the system should be capable of purging memory when it needs to.

2) Continue using your original code, but add memory management to it. That is, when you get create an AVPlayer instance, you will need to account for the size of the video in bytes, and keep a running tally of all this memory. When you approach 50% of total device free memory, then start purging old AVPlayers.

Code

For completeness, I've provided the relevant code from PhotoScrollerNetwork below. If you want more details you can peruse the project - however its quite complex so expect to spend some time (its doing JPEG decoding on the fly for massive images and writing tiles to the file system as the decode proceeds).

// Data Structure
typedef struct {
    size_t freeMemory;
    size_t usedMemory;
    size_t totlMemory;
    size_t resident_size;
    size_t virtual_size;
} freeMemory;

Early on in your app:

// ubc_threshold_ratio defaults to 0.5f
// Take a big chunk of either free memory or all memory
freeMemory fm       = [self freeMemory:@"Initialize"];
float freeThresh    = (float)fm.freeMemory*ubc_threshold_ratio;
float totalThresh   = (float)fm.totlMemory*ubc_threshold_ratio;
size_t ubc_threshold = lrintf(MAX(freeThresh, totalThresh));
size_t ubc_usage = 0;

// Method on some class to monitor the memory pool
- (freeMemory)freeMemory:(NSString *)msg
{
    // http://stackoverflow.com/questions/5012886
    mach_port_t host_port;
    mach_msg_type_number_t host_size;
    vm_size_t pagesize;
    freeMemory fm = { 0, 0, 0, 0, 0 };

    host_port = mach_host_self();
    host_size = sizeof(vm_statistics_data_t) / sizeof(integer_t);
    host_page_size(host_port, &pagesize);        

    vm_statistics_data_t vm_stat;

    if (host_statistics(host_port, HOST_VM_INFO, (host_info_t)&vm_stat, &host_size) != KERN_SUCCESS) {
        LOG(@"Failed to fetch vm statistics");
    } else {
        /* Stats in bytes */ 
        natural_t mem_used = (vm_stat.active_count +
                              vm_stat.inactive_count +
                              vm_stat.wire_count) * pagesize;
        natural_t mem_free = vm_stat.free_count * pagesize;
        natural_t mem_total = mem_used + mem_free;

        fm.freeMemory = (size_t)mem_free;
        fm.usedMemory = (size_t)mem_used;
        fm.totlMemory = (size_t)mem_total;

        struct task_basic_info info;
        if(dump_memory_usage(&info)) {
            fm.resident_size = (size_t)info.resident_size;
            fm.virtual_size = (size_t)info.virtual_size;
        }

#if MEMORY_DEBUGGING == 1
        LOG(@"%@:   "
            "total: %u "
            "used: %u "
            "FREE: %u "
            "  [resident=%u virtual=%u]", 
            msg, 
            (unsigned int)mem_total, 
            (unsigned int)mem_used, 
            (unsigned int)mem_free, 
            (unsigned int)fm.resident_size, 
            (unsigned int)fm.virtual_size
        );
#endif
    }
    return fm;
}

When you open a video, add the size to ubc_usage, and when you close one decrement it. When you want to open a new video, test ubc_usage against ubc_threadhold, and its it exceeds the value you have to close something first.

PS: you can try calling that freeMemory method at other times, and see, but in my case it hardly changes at all when files get opened - the system seems to consider the whole UBC as "free", since it could purge it if it needed to (I guess).

Answer 2

If you're throwing all of these videos in a NSCache, you have to be prepared for the cache to throw away items when it feels like they are consuming too much memory. From the NSCache documentation:

The NSCache class incorporates various auto-removal policies, which ensure that it does not use too much of the system’s memory. The system automatically carries out these policies if memory is needed by other applications. When invoked, these policies remove some items from the cache, minimizing its memory footprint.

Check to see if you're getting nils back from the cache, and if you are, you'll have to reconstruct your objects.

Edit:

It is also worth mentioning that objc.io #7 advises against storing large objects in a NSCache:

The eviction method of NSCache is non-deterministic and not documented. It’s not a good idea to put in super-large objects like images that might fill up your cache faster than it can evict itself.