Unit tests not started with Xcode 5 and jenkins

Question

I've set up jenkins on my local machine. Than I created a sample app with one failing and one succeeding unit test. When I issue the following command in the terminal

xcodebuild -scheme 'SampleWithTest' -sdk iphonesimulator7.0 -destination platform='iOS Simulator',OS=7.0,name='iPhone Retina (3.5-inch)' clean test

than it clean build the project, starts the simulator and makes the tests. This is how it should work.

When I run the same command via jenkins, it clean builds the project and ends with ** TEST SUCCEEDED ** without starting the simulator or print out the test results. Jenkins just fakes, that the tests was successful.

I test it on my local machine. Nothing headless, remote, source control, slave, code signing or whatever could make any trouble.

Answer 1

The Jenkins Xcode Plugin has not been updated to work with Xcode 5 and the XCTest framework. See this issue. Jenkins and the Xcode Plugin do not understand XCTest output.

Additionally, Xcode 5 changed how Xcode interacts with the simulator and devices when running tests. While there is something like a "headless mode" now for the simulator, access to that functionality isn't available to the Xcode plugin. This means that in some cases, it looks to the Xcode plugin as if the tests run and exit immediately, which is not accurate.

Answer 2

Xcode is known to be pretty tricky to setup with Jenkins, Travis or other non-Apple CI solutions.

Unit Tests support is even worse so you should try using xctool instead.

Related blog entry here.