Android NDK mmap call broken on 32-bit devices after upgrading to Lollipop

Question

I'm trying to grab 784 MiB of memory. Yes, I know that is a lot for a 32-bit phone, but the following call worked before Android 5.0:

mmap(0, 0x31000000, PROT_NONE, MAP_ANON | MAP_SHARED, -1, 0);

However, on three different devices from different manufacturers, upgrading to Android 5.0 has broken this. I assume this is some change in memory allocation functionality in 5.0; maybe different flags need to be passed in?

Here's the error message returned in logcat:

E/libc﹕ mmap fail (pid 9994, tid 10125, size 822083584, flags 0x21, errno 12(Out of memory))

Answer 1

At the point where the mmap() fails, open /proc/self/maps and copy the contents to a temp file, then examine the file in an editor. You should see a bunch of entries like this:

12e01000-42c00000 ---p 00201000 00:04 11639      /dev/ashmem/dalvik-main space (deleted)
55281000-5d500000 r--s 00000000 00:16 61         /storage/sdcard1/blah
5d500000-67e80000 rw-p 00000000 00:00 0          [anon:libc_malloc]
67ea4000-682cc000 r-xp 00000000 b3:17 114807     /system/vendor/lib/libsc-a3xx.so
682cc000-682f4000 r--p 00427000 b3:17 114807     /system/vendor/lib/libsc-a3xx.so

The numbers on the left are virtual address ranges (start / end) for the process. When you create a new mapping, it needs to fit in the gap between mappings.

In the example above, there's a nice big gap between the end of the first entry at 0x42c00000 and the start of the next at 0x55281000. That's about 294MB. There's no space between the next two, and only a small one after that.

If you look at your process map, and don't find a gap big enough to hold your file, you have your answer. The region between 0x00000000 and 0xbfffffff is generally available to 32-bit apps, but the app framework uses up a great deal of it. (The top 1GB is mapped to the kernel.)

My guess is that some combination of ASLR and changes to the way virtual memory is allocated in Lollipop have led to this problem. In the map attached to this similar question, the largest gap found was about 300MB. There are two large "dalvik" regions, one 768MB (at 12e01000), one 1.2GB (at 84d81000). (Since you are running Lollipop these are actually due to ART rather than Dalvik, but apparently the label stuck.)

One possibility is that ART has higher virtual memory requirements than Dalvik did, and the large allocations are making it difficult for applications to get large mapped regions. It's also possible that ART is over-allocating due to a bug. You may want to test on Marshmallow to see if something got fixed.

Whatever the case, you can't create a mapping if there isn't a contiguous virtual memory address region large enough to hold it. With the app framework usage and the enormous ART allocations seen in the other question, a 768MB mapping wouldn't be possible even if the virtual address space weren't fragmented. You will need to map smaller sections of the file, and possibly un-map them as you work to make room.

It might be worth filing a bug on b.android.com. Attach a copy of your process map file, and be specific about version of Android and device.

For a bit more about interpreting /proc/maps output, see e.g. this answer.