MonoTouch instability continues: managed memory allocator crashes

Question

Long story short: I can allocate tons of unmanaged memory, but trying to allocate the same amount (or far less) in managed memory crashes MonoTouch in GC_remap (callstack below).

Details:

I'll talk about one example of the behavior I described above. My app allocates a 2.5MB chunk of managed memory (using new byte[]) occasionally, and it often dies on my iPhone4 with the callstack pasted below (i.e. mprotect error during the allocation). I don't keep a reference to these 2.5MB blocks for longer than a single function call.

The MonoTouch guys say that 'mprotect errno 12' means you've exhausted memory on your device, but the thing is, I have lots of memory available to my app. I can allocate 0MB, 10MB, or 200MB of unmanaged memory (using Marshal.AllocHGlobal) at my app's startup, touch it every frame, and it makes zero difference in the behavior of my app or in the frequency of this mprotect error.

Some notes

• GC.TotalMemory tells me that my app is sitting between 3MB and 5MB of managed memory usage all the time.
• I have other places in my app where I'm allocating even-larger blocks of unmanaged memory, and it never crashes there. I have created stress tests that load 4MB of (unmanaged) texture data, hand it to GL, and draw it every frame and the app is rock solid until I start asking for large blocks of managed memory.
• GC.CollectionCount barely ever changes unless I call GC.Collect myself.
• The same behavior happens with MonoTouch 3.2.3 as well as MonoTouch 4.0.
• The same behavior happens across all of our testing devices (iPhone 3G, 3GS, 4, iPad, iPad2).
• The same behavior happens in release builds and debug builds, although it happens more frequently with debug builds.

Ways to provoke the crash

• If I create a thread that loops around calling GC.Collect, then sleeping for 1ms, this makes the crash happen much sooner (i.e. practically immediately if I'm in a debug build).
• Using certain .NET functionality like WebRequest will cause this crash as well. I can only assume that it's allocating big blocks of managed memory somewhere in there as well.

Ways around the crash

There are two ways to reduce the frequency of the crash or to fix it altogether:

1. If I PRE-allocate that 2.5MB chunk of managed memory and then just keep it around for the lifetime of the app, then there's no crash.
2. If I pin the 2.5MB chunk of memory prior to doing anything with it, that seems to help.

Conclusions / Questions

We have yet to achieve full stability in our app due to this issue. This crash (always inside GC_remap) happens in random allocations throughout our app (the 2.5MB example I have here is just the one that I chose to isolate and repro).

Questions:

1. Can I not trust the managed allocator at all?
2. Why is it that I can allocate 200MB of unmanaged memory, but the managed allocator dies when I'm asking for 2.5MB? (Note: It'll die when I ask for 2.5MB even when I haven't allocated the 200MB of unmanaged memory).
3. Why is it that the app is totally fine if I hog that 2.5MB for the lifetime of the app, but if I give it back to the system (and call GC.Collect) and ask for another 2.5MB later, the crashyness is worse! If this really is a low memory condition, shouldn't it be better to give 2.5MB back to the system than to hog it?

Can we even use MonoTouch?

My team is seriously considering abandoning MonoTouch for our product because we can't get it to be reliably stable.

We also can't get the time of day from the MonoTouch team either on stackoverflow, by filing bugs on Novell's site, or by emailing MonoTouch's support email directly. We have reduced our (managed and unmanaged) memory usage to ridiculous lows, but the app is still crashy due to this issue.

In the short term, the only workaround I've got in mind is to allocate a big chunk of memory (2-5MB) at startup, PIN it so the garbage collector never touches it, and write my own allocator to dole out parts of this memory block to my app as needed. But if this is the best solution that is possible under MonoTouch, then I'm going to want my money back as soon as I can achieve escape velocity from MonoTouch.

...

Mprotect failed at 0xaa00000 (length 3801088) with errno 12
Stacktrace:

  at MyApp.GameScreen/VerifyPictureDialog.StoreBasePictureData () [0x00000] in /Users/dussault/s/MyApp/Main/Src/PhotoScreens.cs:428
  at MyApp.GameScreen/VerifyPictureDialog.ApplyFilters (bool) [0x0004b] in /Users/dussault/s/MyApp/Main/Src/PhotoScreens.cs:640
  at MyApp.GameScreen/VerifyPictureDialog.Simulate (single) [0x00077] in /Users/dussault/s/MyApp/Main/Src/PhotoScreens.cs:477
  at MyApp.BaseWindow.Simulate (single) [0x00007] in /Users/dussault/s/MyApp/Main/Src/BaseWindow.cs:56
  at MyApp.BaseWindow.Simulate (single) [0x00007] in /Users/dussault/s/MyApp/Main/Src/BaseWindow.cs:56
  at MyApp.GameScreen.Simulate (single) [0x00238] in /Users/dussault/s/MyApp/Main/Src/GameScreen.cs:3114
  at MyApp.BaseWindow.Simulate (single) [0x00007] in /Users/dussault/s/MyApp/Main/Src/BaseWindow.cs:56
  at MyApp.WindowMgr.Simulate (single) [0x0002f] in /Users/dussault/s/MyApp/Main/Src/WindowMgr.cs:126
  at MyApp.Game1.Update (Microsoft.Xna.Framework.GameTime) [0x0010f] in /Users/dussault/s/MyApp/Main/Src/Game1.cs:1194
  at Microsoft.Xna.Framework.Game.DispatchUpdate (Microsoft.Xna.Framework.GameTime) [0x00000] in /Users/dussault/s/MyApp/Main/Src/XNA-Emulation/GraphicsDevice.cs:531
  at MyApp_iOS.EAGLView.OnUpdateFrame () [0x00050] in /Users/dussault/s/MyApp/Main/Src/iOS/EAGLView.cs:310
  at MyApp_iOS.EAGLView.SimulateAndRender () [0x0000a] in /Users/dussault/s/MyApp/Main/Src/iOS/EAGLView.cs:279
  at MyApp_iOS.EAGLView.MainLoopTimerCallback () [0x00006] in /Users/dussault/s/MyApp/Main/Src/iOS/EAGLView.cs:231
  at MonoTouch.Foundation.NSActionDispatcher.Apply () <0x0002b>
  at (wrapper runtime-invoke) object.runtime_invoke_dynamic (intptr,intptr,intptr,intptr) <0xffffffff>
  at MonoTouch.UIKit.UIApplication.Main (string[],string,string) <0x000cf>
  at MonoTouch.UIKit.UIApplication.Main (string[]) <0x00023>
  at MyApp_iOS.Application.Main (string[]) [0x00000] in /Users/dussault/s/MyApp/Main/Src/iOS/Main.cs:57
  at (wrapper runtime-invoke) object.runtime_invoke_dynamic (intptr,intptr,intptr,intptr) <0xffffffff>

Native stacktrace:

    0   MyApp_iOS                           0x00af1b48 mono_handle_native_sigsegv + 412
    1   MyApp_iOS                           0x00b1c66c sigabrt_signal_handler + 148
    2   libSystem.B.dylib                   0x33bd3ddf _sigtramp + 42
    3   libSystem.B.dylib                   0x33bd52cb kill + 10
    4   libSystem.B.dylib                   0x33bd52bd raise + 16
    5   libSystem.B.dylib                   0x33be9d79 abort + 56
    6   MyApp_iOS                           0x00c74378 GC_remap + 200
    7   MyApp_iOS                           0x00c62c04 GC_allochblk_nth + 1536
    8   MyApp_iOS                           0x00c625b4 GC_allochblk + 96
    9   MyApp_iOS                           0x00c6bf6c GC_alloc_large + 132
    10  MyApp_iOS                           0x00c6c5e8 GC_generic_malloc + 324
    11  MyApp_iOS                           0x00c6c8c8 GC_malloc_atomic + 332
    12  MyApp_iOS                           0x00bd8e88 mono_object_allocate_ptrfree + 64
    13  MyApp_iOS                           0x00bd8ff4 mono_array_new_specific + 148
    14  MyApp_iOS                           0x009173f4 wrapper_managed_to_native_object___icall_wrapper_mono_array_new_specific_intptr_int + 68
    15  MyApp_iOS                           0x002cd880 MyApp_GameScreen_VerifyPictureDialog_ApplyFilters_bool + 628
    16  MyApp_iOS                           0x002cbffc MyApp_GameScreen_VerifyPictureDialog_Simulate_single + 768
    17  MyApp_iOS                           0x002ef9d0 MyApp_BaseWindow_Simulate_single + 280
    18  MyApp_iOS                           0x002ef9d0 MyApp_BaseWindow_Simulate_single + 280
    19  MyApp_iOS                           0x002a71fc MyApp_GameScreen_Simulate_single + 2736
    20  MyApp_iOS                           0x002ef9d0 MyApp_BaseWindow_Simulate_single + 280
    21  MyApp_iOS                           0x0038068c MyApp_WindowMgr_Simulate_single + 376
    22  MyApp_iOS                           0x0027f798 MyApp_Game1_Update_Microsoft_Xna_Framework_GameTime + 1992
    23  MyApp_iOS                           0x0039afc8 Microsoft_Xna_Framework_Game_DispatchUpdate_Microsoft_Xna_Framework_GameTime + 148
    24  MyApp_iOS                           0x0026ec10 MyApp_iOS_EAGLView_OnUpdateFrame + 716
    25  MyApp_iOS                           0x0026e8cc MyApp_iOS_EAGLView_SimulateAndRender + 196
    26  MyApp_iOS                           0x0026e1cc MyApp_iOS_EAGLView_MainLoopTimerCallback + 296
    27  MyApp_iOS                           0x009a7dfc MonoTouch_Foundation_NSActionDispatcher_Apply + 44
    28  MyApp_iOS                           0x00912540 wrapper_runtime_invoke_object_runtime_invoke_dynamic_intptr_intptr_intptr_intptr + 200
    29  MyApp_iOS                           0x00acc9c4 mono_jit_runtime_invoke + 2800
    30  MyApp_iOS                           0x00bd3ea4 mono_runtime_invoke + 140
    31  MyApp_iOS                           0x00c7d214 monotouch_trampoline + 2840
    32  Foundation                          0x3363b469 __NSFireTimer + 136
    33  CoreFoundation                      0x33a770a3 __CFRUNLOOP_IS_CALLING_OUT_TO_A_TIMER_CALLBACK_FUNCTION__ + 14
    34  CoreFoundation                      0x33a76b5b __CFRunLoopDoTimer + 850
    35  CoreFoundation                      0x33a481b5 __CFRunLoopRun + 1088
    36  CoreFoundation                      0x33a47c87 CFRunLoopRunSpecific + 230
    37  CoreFoundation                      0x33a47b8f CFRunLoopRunInMode + 58
    38  GraphicsServices                    0x33b0e4ab GSEventRunModal + 114
    39  GraphicsServices                    0x33b0e557 GSEventRun + 62
    40  UIKit                               0x32099329 -[UIApplication _run] + 412
    41  UIKit                               0x32096e93 UIApplicationMain + 670
    42  MyApp_iOS                           0x009d484c wrapper_managed_to_native_MonoTouch_UIKit_UIApplication_UIApplicationMain_int_string___intptr_intptr + 240
    43  MyApp_iOS                           0x009b4c00 MonoTouch_UIKit_UIApplication_Main_string__ + 36
    44  MyApp_iOS                           0x00269694 MyApp_iOS_Application_Main_string__ + 128
    45  MyApp_iOS                           0x00912540 wrapper_runtime_invoke_object_runtime_invoke_dynamic_intptr_intptr_intptr_intptr + 200
    46  MyApp_iOS                           0x00acc9c4 mono_jit_runtime_invoke + 2800
    47  MyApp_iOS                           0x00bd3ea4 mono_runtime_invoke + 140
    48  MyApp_iOS                           0x00bd6f3c mono_runtime_exec_main + 784
    49  MyApp_iOS                           0x00bd5f6c mono_runtime_run_main + 1048
    50  MyApp_iOS                           0x00ad7940 mono_jit_exec + 216
    51  MyApp_iOS                           0x00ac2e38 main + 3536
    52  MyApp_iOS                           0x000133a0 start + 52

Debug info from gdb:


=================================================================
Got a SIGABRT while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries
used by your application.
=================================================================

Answer 1

The Darwin kernel overcommits memory.

What this means is that if you request 200 megs of ram, you will get them, even if they are not available and as long as you do not actually consume the memory, your application will run fine.

Only when you actually write to the page will the page be allocated to your process.

A proper test would require you to allocate the memory and then fill up the used memory, this is why you probably get the impression that you can allocate 200 megs of ram even when you might not even have it.

A simple program will show you this: try allocating 500 megs, the OS will say "Yes, you got it", but there are no iPhones with this kind of memory.

A sample test case would go a long way to showing what the problem is.