Performance Testing tools for native mobile applications in iOS and Android [closed]

Question

I have to do performance testing of native applications on the iOS and Android platforms. The current requirements from the tool are:

- The tool needs to check for the native applications CPU and memory usage.
- If possible, we need to provide if there are any memory leaks also involved with the native application.
- Check for network performance over the various networks - EDGE, 2G, 3G, 4G and wireless connections (various speeds).
- If we can do Functional testing with the same application, it would be a great addition.

What I have been able to figure out is the following:

1.  UI Response Time
◦   Download times for intermediate action requests (patch or network downloads), to be reduced with better compression techniques
◦   User action and response rates of the application, need to be smoothened by employing better software rendering and faster animations
◦   Provide a progress indicator to the user, along with completion notification
2.  Battery Life
◦   Battery drain is non-linear, so minimize radio wake-up calls and
◦   Use system events, instead of polling
3.  Network Bandwidth
◦   Basically depends on - signal strength, carrier networks and network type (performance can be considered for slower networks, but no guarantee on the faster 2G & above).
◦   Faster networks need to be checked basically for the functionality
4.  Memory and CPU
◦   Overall device sluggishness needs to be monitored with respect to the response of the various actions. This can only be monitored by running the functional tests, and checking the actual RAM and CPU counters using a recording tool.
◦   No application crashes should be seen, even after keeping the application open for a few days.
5.  Binary Size
◦   Effects application load time
◦   Effects load RAM size
◦   Download & install times
6.  Tools
◦   http://spb.com/pocketpc-software/wirelessmonitor/
◦   http://mobitest.akamai.com/m/index.cgi
◦   http://www.neotys.com/product/mobile-load-testing.html

 

My question is are there any other parameters which we can add to the above to test the Performance of a native application?

Also, which other Open Source and Commercial tools would you suggest, which fulfils all the above criteria for performance testing of a native mobile application on the iOS, Android and WindowsOS platforms?

Answer 1

That's a very comprehensive write up when it comes to performance testing of mobile apps. I have a couple of small things to add: Regarding memory: 1. You would probably also want to monitor peak memory consumption at any instant. This is especially important on Android because the available memory for apps may actually vary from device to device. And hence, even though it may not cause problems for you on the device on which test is being conducted, it may cause issues on some other lower-end device.

1. In a similar vein, you probably would also want to monitor garbage collections happening and how many are concurrent and how many are alloc based gc's. Because gc's can potentially affect app response times, especially if it is caused for an alloc.

Regarding battery life: 1. You may want to verify that any wakelocks acquired by the app are being released promptly. 2. The app is using inexact timers as much as possible instead of precise timers (e.g., use setInexactRepeating instead of setRepeating method of AlarmManager).

Some other criterion: 1. Verify that the app is caching data appropriately - not refreshing content it already has when the screen orientation changes, for instance. 2. And even better apps, may actually condition their data consumption based on the type of network connected - go crazy and prefetch stuff if connected to Wifi, but fetch a little less aggressively if connected on cellular data.

Hope this helps.