XCTests Failing on Physical Device: "Canceling tests due to timeout ..."

Question

XCTests are failing with the message: *** Canceling tests due to timeout in Waiting for test process to check in... This just started coming up in the last few days. I'm using Xcode 7.3.1, with iOS 9.3.2 running on an iPhone 6. My app is written mostly in Swift.

I've seen some similar posts:

1. Unable to run XCTests on iOS device
2. iOS tests will not run on simulator when using Jenkins through JNLP

Those other posts talk about this issue as arising with code signing. Code signing does not seem to be my problem-- I've looked at the KeyChain Access utility and don't see any relevant expired certificates. Also, what so far is (very temporarily) fixing my problem is to restart my iPhone. (Unfortunately, that fix doesn't last long-- maybe a few runs of XCtests and the problem is coming up again). I'm not running Jenkins, just XCTests.

I have tried restarting Xcode and removing all files/folders from the DerivedData folder but neither of these resolves the problem.

I did just install Xcode8 (first beta release). But aside from having launched it once or twice, I'm not generally using it. It does seem oddly coincidental that this issue is arising after having just installed that.

Update 6/25/16

I've narrowed this issue down somewhat. Several other symptoms appear at the same time as the timeout issue:

1. Console logging directly from print statements in the XCTest files stops.
2. Timer's fail-- this is actually the root of the problem. My tests involve first waiting for some server interaction, which occurs. But the wait uses an NSTimer which never executes its callback.
3. Breakpoints stop working.

It seems worthwhile to note that I'm running these XCTest's manually. That is, I am running each test separately, so each test involves a build.

Also, so far I have tested the following:

1. Restarting Xcode (doesn't help)
2. Restarting Mac OS X (doesn't help)
3. Removing Derived Data contents (doesn't help)
4. Restarting iPhone-- helps, but only again allows a few Xcode tests to run.
5. Tried running with wifi versus hotspot on iPhone (no change to issue)
6. TODO: Running with simulator
7. Could this be a cable or USB port issue? Changing the cable connecting the device doesn't help.
8. Deleting the app and reinstalling/rebuilding doesn't help.
9. Tried on different hardware (iPad Air running iOS 9.3.2). Same issues.

My configuration is: iOS 9.3.2, Xcode 7.3.1, Mac OS X 10.11.5 (15F34).

Answer 1

The problem lies in the fact (or rather: serious bug in xcodebuild) that the timeout for connecting to the XCTest server starts at the moment you issue the command xcodebuild. The timeout is 120 seconds, so if your compilation + startup of the simulator takes longer than 2 minutes xcodebuild will give this "Canceling tests due to timeout" error.

The solution is to break up the build into two commands. One for building and one for running the tests:

> xcodebuild clean build build-for-testing <other options>
> xcodebuild test-without-building <other options>

This will solve the timeout issue, because the test-without-building action doesn't have to compile first.