Benchmarking cold startup times

Question

I want to be able to measure cold app startup times locally in a benchmark test (median time, without too much jitter). Does anybody know if with the jetpack benchmark library we can measure cold app startup times reliably? If so, how? I know that, with firebase performance plugin, we can measure some median startup times, but I want to measure locally those improvements.

I only can think of a script that launches through adb the app multiple times and measures the first ActivityManager: Displayed from the logcat...

Answer 1

I believe you can achieve pretty accurate results using android profiler, for example here (picture below), it took 4.233 sec to launch the main activity. I think the hard part will be to parse the profiler results, you can read more about it here: https://android.googlesource.com/platform/system/extras/+/master/simpleperf/doc/README.md

but note, the app startup time could be very different between devices, configuration, os status etc... so I think the best approach will be to measure it on a big variety of devices (using a tool like firebase performance) I think this is the only way you can get a useful insights.

Answer 2

You can use Jetpack Macrobenchmark library that allows benchmarking exactly what you described. You can tweak startup mode and even the compilation state of your app. Each benchmark saves system traces on the device and to Android Studio output, so it's easy to dig in to what's hapenning during startup.

Jetpack Benchmark library (renamed to Jetpack Microbenchmark) is better suited for benchmarking hot code (JITted, disk-cached, optimized).