UIAutomation with code coverage

Question

I am currently automating our iOS testing on jenkins. So far I had no problems with running unit tests, converting OCUnit into JUnit and generating code coverage in Cobertura format (lots of googling but no hard problems).

However, when creating UIAutomation job, I am stuck. The tests are running successfully (calling instruments from command line). Generating junit report was tricky but possible. The problem is that when running UIAutomation, no code coverage files are generated.

Is there a possibility to generate them? If not, could you please explain why?

What I tried so far:

1. Attaching UIAutomation to an already running application on iOS simulator.
   • this seems impossible. Either the running application is killed by instruments and a new instance is relaunched or a cryptic error message is printed.
2. Quit simulator at the end of js script using a combination of osascript and UIAHost. performTaskWithPathArgumentsTimeout.
   • application ended gracefully but no coverage generated.

Edit After some testing with a different framework I realized it's not enough to quit the simulator, you have to explicitely call exit() from the application. With UI Automation this is a bit tricky but you can declare an applicaton scheme, e.g. my-app://exit and call it through MobileSafari using UIAHost.performTaskWithPathArgumentsTimeout. Will check whether this is enough for the files to be generated.

Answer 1

Following these steps, I was able to generate the code coverage files from UI Automation and display the information through the cobertura Jenkins plugin.

First set the “Generate Test Coverage Files” and “Instrument Program Flow” build settings to Yes. This will generate code coverage files every time you run your application in the simulator and exit the application. Add UIApplicationExitsOnSuspend in your Info.plist file and set this option to 'YES'. Run the UI automation test and at the end of it you can exit the app either by manually pressing the HOME button in the simulator or using the UIATarget.localTarget().deactivateAppForDuration() method. Note if your app has any UI Automation tests that rely on the deactivateAppForDuration() method, the tests will terminate upon running the command.

Once you have the gcda files you can generate the cobertura xml file by downloading gcovr (https://software.sandia.gov/trac/fast/wiki/gcovr) and running the command

gcovr -r your_root_directory --object-directory path_to_gcda_files --xml > coverage.xml

With that you can setup the Jenkins cobertura plugin to display the information as needed.

Source: http://blog.octo.com/en/jenkins-quality-dashboard-ios-development/#step2-2