Why do large Android Activity Unit-tests fail?

Question

I'm trying to create a unit-test for an Activity with a lot of methods to test. But after about 31 tests the application gets killed because the heap is out of memory.

1152 E SurfaceFlinger   createSurface() failed, generateId = -12
1152 W WindowManager    OutOfResourcesException creating surface
1152 I WindowManager    Out of memory for surface!  Looking for leaks...
1152 W WindowManager    No leaked surfaces; killing applicatons!
1152 W ActivityManager  Killing processes Free memory at adjustment 1

I have made a unit-test with 40 identical simple test-cases just to find the problem. But it feels like the GC is not fast enough to cleanup the memory during tests.

Here is my leakTest test-case:

package my.app;

import android.os.Debug;
import android.test.ActivityInstrumentationTestCase2;
import android.util.Log;

public class leakTest extends
        ActivityInstrumentationTestCase2<TestActivityAndroid> {

    String TAG = "leakTest";

    TestActivityAndroid mActivity = null;

    public leakTest() {
        super(TestActivityAndroid.class);
    }

    protected void setUp() throws Exception {
        super.setUp();
        setActivityInitialTouchMode(false);

        mActivity = getActivity();
    }

    protected void tearDown() throws Exception {
        super.tearDown();
    }

    private void printHeapSize() {
        Log.e(TAG,
                "NativeHeapAllocatedSize = "
                        + Debug.getNativeHeapAllocatedSize());
        Log.e(TAG, "NativeHeapFreeSize = " + Debug.getNativeHeapFreeSize());
        Log.e(TAG, "NativeHeapSIZE = " + Debug.getNativeHeapSize());
    }

    public void test_1() {
        assertNotNull(mActivity);
    }

    public void test_2() {
        assertNotNull(mActivity);
    }

    public void test_3() {
        assertNotNull(mActivity);
    }

    public void test_4() {
        assertNotNull(mActivity);
    }

    public void test_5() {
        assertNotNull(mActivity);
    }

    public void test_6() {
        assertNotNull(mActivity);
    }

    public void test_7() {
        assertNotNull(mActivity);
    }

    public void test_8() {
        assertNotNull(mActivity);
    }

    public void test_9() {
        assertNotNull(mActivity);
    }

    public void test_10() {
        assertNotNull(mActivity);
    }

    public void test_11() {
        assertNotNull(mActivity);
    }

    public void test_12() {
        assertNotNull(mActivity);
    }

    public void test_13() {
        assertNotNull(mActivity);
    }

    public void test_14() {
        assertNotNull(mActivity);
    }

    public void test_15() {
        assertNotNull(mActivity);
    }

    public void test_16() {
        assertNotNull(mActivity);
    }

    public void test_17() {
        assertNotNull(mActivity);
    }

    public void test_18() {
        assertNotNull(mActivity);
    }

    public void test_19() {
        assertNotNull(mActivity);
    }

    public void test_20() {
        assertNotNull(mActivity);
    }

    public void test_21() {
        assertNotNull(mActivity);
    }

    public void test_22() {
        assertNotNull(mActivity);
    }

    public void test_23() {
        assertNotNull(mActivity);
    }

    public void test_24() {
        assertNotNull(mActivity);
    }

    public void test_25() {
        assertNotNull(mActivity);
    }

    public void test_26() {
        assertNotNull(mActivity);
    }

    public void test_27() {
        assertNotNull(mActivity);
    }

    public void test_28() {
        assertNotNull(mActivity);
    }

    public void test_29() {
        assertNotNull(mActivity);
    }

    public void test_30() {
        assertNotNull(mActivity);
    }

    public void test_31() {
        assertNotNull(mActivity);
    }

    public void test_32() {
        assertNotNull(mActivity);
    }

    public void test_33() {
        assertNotNull(mActivity);
    }

    public void test_34() {
        assertNotNull(mActivity);
    }

    public void test_35() {
        assertNotNull(mActivity);
    }

    public void test_36() {
        assertNotNull(mActivity);
    }

    public void test_37() {
        assertNotNull(mActivity);
    }

    public void test_38() {
        assertNotNull(mActivity);
    }

    public void test_39() {
        assertNotNull(mActivity);
    }

    public void test_40() {
        assertNotNull(mActivity);
    }
}

The test activity which extends Activity:

package my.app;

import android.app.Activity;

import android.os.Bundle;

public class TestActivityAndroid extends Activity {

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
    }
}

Here is the memory usage where the native free space drops to below 30kB and than the app is killed.

        Applications Memory Usage (kB): Uptime: 3804373 Realtime: 3804373

** MEMINFO in pid 7315 [my.app] **
                    native   dalvik    other    total
            size:     4048     3271      N/A     7319
       allocated:     3942     2306      N/A     6248
            free:      105      965      N/A     1070
           (Pss):      844     1590     1806     4240   
  (shared dirty):     1404     4120     2288     7812
    (priv dirty):      736      672      992     2400    Objects
           Views:        0        ViewRoots:        0
     AppContexts:        0       Activities:        0
          Assets:        2    AssetManagers:        2    
   Local Binders:       11    Proxy Binders:       10 
Death Recipients:        0      
OpenSSL Sockets:         0    
SQL
            heap:        0       memoryUsed:        0 
pageCacheOverflo:        0  largestMemAlloc:        0
     Asset Allocations
    zip:/data/app/my.app-1.apk:/resources.arsc: 1K

Does somebody have a better solution that the 2 sec sleep inside the tearDown()? I don't like the sleep inside the tearDown(). And because we have around 100 tests inside our test-suite the 2 sec will be a huge delay.

I hope somebody can help me and if my question is not clear please let me know.

Thanks in advance.

Answer 1

Why do you need to do a gc after each Unit test?

If it is because you want a clean environment for your test, then live with the 2 second delay. tearDown does at least 2 gc's and some finalization. It leaves a clean environment for your next test. If you read through the android source code, there are several comments indicating the critical need of callilng tearDown at the end of the test.

If your tests don't need a clean environment, then combine them. That being said, 200 seconds of a process running behind the scenes is a small price to pay for the protection the tests will give you.

Our automated tests on our current big project take about 5 minutes to run. I never notice, because we have automated our system to run the tests at initial check-in and bounce the submission if they fail.

The few times they have failed, I've been really startled that the code I changed messed up other sections of the app. We've saved at least weeks, and likely many months in maintenance and debugging our app with our automated tests.