Xcode 11 XCUITest Failed to get matching snapshots: Error getting main window kAXErrorServerNotFound

Question

After building my app in Xcode 11 and running my suite of XCUITests I am getting many random failures with the following.

Failed to get matching snapshots: Error getting main window kAXErrorServerNotFound

No matter how long I increase timeouts the issues pop up intermittently. It seems to be having issues Snapshotting the UI hierarchy. Our tests pass consistently in Xcode 10.

I have reinstalled Xcode. Deleted all simulators. Cleared derived data. Modified timeouts. Upgraded from Xcode 11.1 to Xcode 11.2.1.

Thanks!

 

Answer 1

I had the problem with matching while I was running the simple UITest in Xcode 11.3. To make it work I had to paste at first: XCUIApplication().activate() or XCUIApplication().launch().

Answer 2

I have experienced the same issue with Xcode 11 and realized that the test runner was not getting killed when stopping the tests (or if the test crashed for some reason). Running the tests a second time would spawn a new test runner and at that point I had two runners trying to interact with the same application, leading to this very strange error.

To prove that I did the following:

1. Created a UI test that types a long text in a text view
2. Ran the test, and manually stopped it when there were a few sentences in the text view
3. Manually opened the app in the simulator (not by running the test)
4. Observed that random characters were appearing in the text view, even though no tests were running.

The workaround was to quit and reopen the simulator to make sure all processes were getting killed. Hope this solves your issues

Answer 3

I meet the issue from time to time in Xcode 11.1. I observed that the issue happen when waiting for UI elements especially there are web view being shown during the test. When the issue happened I was using XCUIElement.waitForExistence(timeout:) or expectation with NSPredicate(format: "exists == true"). When I changed to use expectation with NSPredicate(format: "hittable == true") the issue seems to be gone but I don't know why. The difference between the 2 attributes is that hittable only detect onscreen elements while exists detect off-screen elements such as off-screen cells of a table view.

expectation(for: NSPredicate(format: "hittable == true"), evaluateWith: element, handler: nil)
waitForExpectations(timeout: 60, handler: nil)