How to test android enterprise application on various devices

Question

I am working on a big enterprise android application and we are wondering how are the other companies testing their apps. We have about 10 kinds of devices, which we recommend to our customers, because it is impossible to have all kinds of devices and test app on them.

I heard that Eset and Sygic have also some main supported devices on which is their app tested and in case some device specific bug, they just get that device.

I made some research and I found some automated testing tools (like TestDroid). Is anyone having real experience with automated testing tools like TestDroid?

It would be awesome, if someone who is working for some big company could share testing procedures, tools, hints or recommendations how to test enterprise android applications when exists so many devices with various OS versions, resolutions and system extensions (like HTC Sense, Samsung TouchWiz).

Thanks a lot guys.

Answer 1

Below information is conclusion of our experiences;

The best way of supporting large scale of device set is testing your app with your own testing devices as you do. For big enterprise project you should think to cover most of the current user if the company hasn’t specific device group.

There is a way of deciding top devices by below method;

You should check android stats dashboard page for retrieving info for understanding most using device specs. Simple way of doing this is by taking the combination of investigating stats of Platform Versions (according To Distributions) && Screen Sizes and Densities.

Then you have a rough list of popular devices and you should own as much as of them.

If your budget doesn’t enough to buy lots of device you may choose to outsource testing from some companies like testDroid etc. But I doNOT recommend that because of the general automated test what have done is not enough for your app and when there is an itchy bug pops up you should have this device to be sure that is fixed.

Although We decide to use our devices for testing, We have tried outsourcing in past, and there are top 3 of our choice;

• #1 Apkudo https://www.apkudo.com/developers.html
• #2 Appthwack https://appthwack.com
• #3 If you have an good testing w/ has a passion to make automation, I can refer Ranorex. You can predefined test flow according to your features||testCases and it has pretty promising ui and feedback system. http://www.ranorex.com/mobile-automation-testing/android-test-automation.html

Answer 2

Actually, there are plenty of tools for automated testing of Android apps. Take a look at a small overview of the most popular tools that Android testers use for automated testing: http://blog.bughuntress.com/automated-testing/useful-tools-for-android-apps-test-automation. Looking through them, you’ll probably find something useful. They all are commonly used for test automation in big companies.

Answer 3

if you part of development team, and QA most likely from some other team you might end up with two sets of test cases. You have to proof to your manager that you done your job and that passes test cases and QA will do their part of testing to show their manager this app pass all the requirements.

I think our QA team is using couple of these services mention here from time to time but not all times. When, i asked i have been told it is not very convenient for them to use these server as it is very time consuming, and their are cases where all of them failed to test certain thing.

But on our side(dev team) we didn't have extra budget and didn't get satisfactory go ahead from QA and also from the developer who studied these services side by side with our requirement. So, we end up doing our own test automation. Which took exactly two days.

1. Using GIT,
2. Jenkins
3. ADB
4. Batch/Shell script.

So whenever we complete a feature and its unit test cases we mark that feature as complete on issue tracker. Developer push the code to a version control system.

Next morning when a developer come to work, he or she go to Jenkins panel and start a build. Jenkins just execute a batch file which does following,

1. download latest code from GIT. uses a git command.
2. Update version id.
3. Execute Android command line build process to build the app.
4. follow the same steps for Unit test application.
5. Install both application on a device or devices connected to that server.
6. Run test application using ADB on those device(s).
7. Collect Unit Test Log(Junit Style).
8. Mail the log to all interested parties.

This is also same for stress testing. But, if you do over night stress testing it might collect megabytes of log, in that case key word base search is definitely a good idea to find crashes.

For layout/resolution testing, you can always take screen shot using adb or from unit test app and attach those images as email attachments as well.

Definitely using a third party service will ease the task and we can always outsource whatever we need. But remember their are cases when manual testing absolutely necessary. Like, if your app want to activate WiFi or anything from Android Settings which require user explicit input, or the cases where you are using another resource, like using camera for taking pictures or testing social network integration. Make sure to compare your requirements with the service those business entities are offering.