Android UI tests with Espresso + MockK crash with SIGSEGV on emulators, fine on physical devices

Question

I have just started using MockK to mock all the Repositories / Services logic in an MVP-based app for UI tests.

I have some UI tests running a login activity where the Espresso inputs logins and passwords and using MockK I can fake various situation where the login fails or not.

All services & repositories are standard Kotlin object, so I am using mockkobject and every/coEvery to override and handle login requests & tasks.

On my physical devices, there are no issues with the tests at all, but as soon as I tried to run them on a emulator running Android P+ with recommended image, they constantly crash at random time. And on rare occasion they can survive long enough to work.

Looking at the logs, I got various SIGSEGV:

A/libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0 in tid 10425 (e.android.debug), pid 10425 (e.android.debug)

A/libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xc in tid 10968 (HeapTaskDaemon), pid 10957 (e.android.debug)

A/libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0 in tid 15050 (firebase-instal), pid 14972 (e.android.debug)

A/libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xd in tid 8902 (Measurement Wor), pid 8858 (e.android.debug)

A/libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0 in tid 22004 (Binder:21832_5), pid 21832 (e.android.debug)

---

But looking deeper into the logs I believe I found the culprit:

InputDispatcher: channel '9fa7335 my.company.com.android.debug/my.company.com.ui.login.LoginActivity(server)' ~ Channel is unrecoverably broken and will be disposed!

Looking for solutions, it seems this could happen due to memory leaks ?

In any case, I made sure to launch the activity under test in a @Before method where all the mocks occur while I am clearing such mocks and verifies in the @After method.

Clearly I believe the tests do work just fine but something must be wrong with Espresso or all the mocking that occurs...

[EDIT 1]

Further looking at prior logs, this could be the reason why there are memory leaks:

ActivityThread: Service com.google.android.gms.autofill.service.AutofillService has leaked IntentReceiver com.google.android.gms.autofill.smsretriever.TracingSmsBroadcastReceiver@ce00658 that was originally registered here. Are you missing a call to unregisterReceiver()? android.app.IntentReceiverLeaked: Service com.google.android.gms.autofill.service.AutofillService has leaked IntentReceiver com.google.android.gms.autofill.smsretriever.TracingSmsBroadcastReceiver@ce00658 that was originally registered here. Are you missing a call to unregisterReceiver()?

I disabled the AutoFillService (set to none in the relevant settings section) on my emulator. This seemed to improve my tests success rate at the beginning, but they keep crashing after several runs. Now the logs are not showing this leak anymore, though.

[EDIT 2]

Apparently this issue could be related to MockK as I was able to retrieve more logs:

2020-07-24 11:57:15.955 15767-15780/com.my.company.android.debug A/libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0 in tid 15780 (HeapTaskDaemon), pid 15767 (e.android.debug)
2020-07-24 11:57:15.997 15962-15962/? E/crash_dump32: failed to detach from thread 15773: No such process
2020-07-24 11:57:15.997 15962-15962/? E/crash_dump32: failed to detach from thread 15778: No such process
2020-07-24 11:57:15.997 15962-15962/? E/crash_dump32: failed to detach from thread 15779: No such process
2020-07-24 11:57:15.997 15962-15962/? E/crash_dump32: failed to detach from thread 15780: No such process

// 20 more times of exact same line //

2020-07-24 11:57:16.007 15962-15962/? A/DEBUG: *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
2020-07-24 11:57:16.007 15962-15962/? A/DEBUG: Build fingerprint: 'google/sdk_gphone_x86/generic_x86:10/QSR1.190920.001/5891938:user/release-keys'
2020-07-24 11:57:16.007 15962-15962/? A/DEBUG: Revision: '0'
2020-07-24 11:57:16.007 15962-15962/? A/DEBUG: ABI: 'x86'
2020-07-24 11:57:16.008 15962-15962/? A/DEBUG: Timestamp: 2020-07-24 09:57:16+0000
2020-07-24 11:57:16.008 15962-15962/? A/DEBUG: pid: 15767, tid: 15780, name: HeapTaskDaemon  >>> com.my.company.android.debug <<<
2020-07-24 11:57:16.008 15962-15962/? A/DEBUG: uid: 10136
2020-07-24 11:57:16.008 15962-15962/? A/DEBUG: signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0
2020-07-24 11:57:16.008 15962-15962/? A/DEBUG: Cause: null pointer dereference
2020-07-24 11:57:16.008 15962-15962/? A/DEBUG:     eax 00000000  ebx ef8a6c34  ecx 00000000  edx f310b604
2020-07-24 11:57:16.008 15962-15962/? A/DEBUG:     edi f3200380  esi 00000000
2020-07-24 11:57:16.008 15962-15962/? A/DEBUG:     ebp e659d9a8  esp e659d940  eip ef7d89f4
2020-07-24 11:57:16.027 2044-2135/? E/InputDispatcher: channel '342ebda com.my.company.android.debug/com.my.compan.android.ui.error.LoginActivity (server)' ~ Channel is unrecoverably broken and will be disposed!

After further investigation, there is a 1-year-old issue on the Android Test Github repo showing the same problem (but issue is now closed): https://github.com/android/android-test/issues/352

A relevant issue at Mockk was opened here: https://github.com/mockk/mockk/issues/466

[EDIT 3]

I was exploring the alternative to go back to Mockito which has more history and more active development. It took a bit of time but I had no major issue migrating my UI tests to Mockito.

The results: well, at first the crash were completely gone. I could fire my tests run, 10, 20, 30 times without a hinch. At least on mobile.

However, on Android TV (still with a simulator), the crash reappeared rather quickly. And then on mobile as well, but way less often with the dreadful message from the InputDispatcher.

While setting up the migration to Mockito, I have noticed that Mockk shares the same restrictions & dependencies with Mockito when mocking on Android Test Instrumentation. I faced the same issues and the same difficulties.

So it led me to believe that neither of them are the culprit, but it could very well be the Android Instrumentation APIs.

I also noticed that manually rebooting the emulators greatly improved the situation.

Answer 1

This was in my stacktrace:

  W  Unexpected CPU variant for X86 using defaults: x86
         ActivityThread  W  Package uses different ABI(s) than its instrumentation:

Switching to a 64 bit emulator fixed the problem.

Answer 2

I had exactly the same case where my tests were crashing randomly on the Emulator with Process crashed error saying check the Logcat. There was SIGSEGV error on the Logcat but not always.

In my case it was com.linkedin.dexmaker:dexmaker-mockito-inline causing the random crashes. I had to add it to mock a Kotlin data class in Instrumentation tests which I didn't need after all. So removing this dependency solved my issue.

I recommend to investigate after which point your tests started crashing, then it will be easier to get to bottom of it.

Answer 3

I too have been running into this issue with my instrumentation test suite and also suspect MockK to be (at least partially) at fault.

This is far from a true solution, but I've noticed that running MY tests with Android Test Orchestrator greatly reduces the chance of random failure due to test process crash (caused by the above segfault). (If you are running tests with Firebase Test Lab emulators, this could be particularly helpful.) Another bonus of using Orchestrator, it does a really good job of digging through the logs (in Firebase Test Lab) to find the root error if your test does fail in an Orchestrator process.

I'm not certain this will work well for everyone, but if it does, it likely indicates that MockK (if it truly is the source of this issue) isn't cleaning up fully and is leaking into other tests.