Measuring performance using Android instrumentation test

Question

My goal was to write automatic performance test for Android CPU intensive code using an instrumental test (AndroidJUnitRunner).

I was very surprised to find that the test results are not reliable, to simulate CPU intensive code, I want to test, I wrote the following loop

for(int i=0;i<1000000;i++){
    Math.pow(2,i);
}

The code was tested as an instrumental test and within an Android app

The result I got was as follows:

Instrumental test showed ~230ms to complete the loop whereas the same code on the same device (G5) took ~600ms

I will appreciate any clue why the execution of same code on AndroidJUnitRunner takes three times less time than on the real device while both of them finally are executed on the same device

Answer 1

Simply speed comes from CPU, while you are executing some code, if CPU is not doing any heavy work and all CPU cores are up, it will execute your code pretty fast. In android, 'UI rendering' is most intensive work that AndroidJUnitRunner is not doing, that's why it is fast.

If you want to understand how android perform in different scenarios, take a look at this: https://www.youtube.com/playlist?list=PLWz5rJ2EKKc9CBxr3BVjPTPoDPLdPIFCE