How can I automatically test the functionality of iOS and Android applications?

Question

I have to regularly test the availability and functioning of a movie rental website. I wrote a Windows program which is able to automate a web browser according to a script, so this task is basically solved. Now I have to automate the mobile version of this web application: a native iOS app and a native Android app.

These apps are closed source, so cannot be modified in any way. I think the test app should be deployed on the test devices (iPhone, iPad, Galaxy Tab, Galaxy S II), but I must be able to remote control it. I mean, I would like create a connection between the test devices and a PC, upload test scripts from the PC to the devices, run them, and download the test results to the PC. The test script should start the app to be tested, manipulate its GUI (fill editboxes, push buttons etc.), and follow its response somehow, for example by analyzing the GUI (the existence of some GUI elements, their caption, etc.), analyzing screenshots, and/or inspecting IP packets.

I wrote lots of similar test programs for Windows: I used ShellExecute, PostMessage, FindWindow, the WinPcap library etc., so I know how such a program should work. But since I never wrote applications for mobile OS's, I don't even know whether there are similar APIs and libraries for iOS and Android.

I would like to know where to start, I mean, which SDKs and developer tools could be used to write such an application. I'm also interested in commercial solutions. I would really appreciate any help.

Answer 1

I like "Calabash-iOS/Calabash-Android" by LessPainful. That is the best for me.

• free
• available on iPhone and Android
• record and playback
• test on native and simulator

They doesn't have a GUI IDE. They are Ruby-based solutions and it is very easy to write test case script, like this:

Then I swipe left
And I wait until I don't see "Please swipe left"
And take picture

Also it can be

Then I touch the "login" button

to tap a button, or

Then I fill in "placeholder" with "text to write"

to write something to a textbox.

And the script can be shared by Calabash-iOS and Calabash-Android. That feature is convenient for developers who make both iPhoneApp and Android App.

I thought MonkeyTalk was nice but actually unstable, at least in my environment.
Sometimes MonkeyIDE crashed, so we have to do debug of MonkeyTalk...

I hope this helps you.