Is it possible to use Xcode UI Testing on an app using SpriteKit?

Question

I want to use UI testing for a game using SKSpriteKit.

As my first tries did not work I wonder if it possible to use Xcode UI Testing with SpriteKit.

Answer 1

The main idea is to create the accessibility material for elements that you want to UI test. That's mean:

1. List all accessible elements contained in the scene

2. Configure settings for each of these elements, especially framedata.

Step by Step

This answer is for Swift 3 and is mainly based on Accessibility (Voice Over) with Sprite Kit

Let's say I want to make the SpriteKit button named tapMe accessible.

List of accessible elements.

Add an array of UIAccessibilityElementto the Scene.

 var accessibleElements: [UIAccessibilityElement] = []

Scene's cycle life

I need to update two methods: didMove(to:)and willMove(from:).

override func didMove(to view: SKView) {
    isAccessibilityElement       = false
    tapMe.isAccessibilityElement = true
}

As scene is the accessibility controller, documentation stated it must return False to isAccessibilityElement.

And:

override func willMove(from view: SKView) {
    accessibleElements.removeAll()
}

Override UIAccessibilityContainer methods

3 methods are involved: accessibilityElementCount(), accessibilityElement(at index:) and index(ofAccessibilityElement. Please allow me to introduce an initAccessibility() method I'll describe later.

override func accessibilityElementCount() -> Int {
    initAccessibility()
    return accessibleElements.count
}

override func accessibilityElement(at index: Int) -> Any? {

    initAccessibility()
    if (index < accessibleElements.count) {
        return accessibleElements[index]
    } else {
        return nil
    }
}

override func index(ofAccessibilityElement element: Any) -> Int {
    initAccessibility()
    return accessibleElements.index(of: element as! UIAccessibilityElement)!
}

Initialize accessibility for the Scene

func initAccessibility() {

    if accessibleElements.count == 0 {

        // 1.
        let elementForTapMe   = UIAccessibilityElement(accessibilityContainer: self.view!)

        // 2.
        var frameForTapMe = tapMe.frame

        // From Scene to View
        frameForTapMe.origin = (view?.convert(frameForTapMe.origin, from: self))!

        // Don't forget origins are different for SpriteKit and UIKit:
        // - SpriteKit is bottom/left
        // - UIKit is top/left
        //              y
        //  ┌────┐       ▲
        //  │    │       │   x
        //  ◉────┘       └──▶
        //
        //                   x
        //  ◉────┐       ┌──▶
        //  │    │       │
        //  └────┘     y ▼
        //
        // Thus before the following conversion, origin value indicate the bottom/left edge of the frame.
        // We then need to move it to top/left by retrieving the height of the frame.
        //


        frameForTapMe.origin.y = frameForTapMe.origin.y - frameForTapMe.size.height


        // 3.
        elementForTapMe.accessibilityLabel   = "tap Me"
        elementForTapMe.accessibilityFrame   = frameForTapMe
        elementForTapMe.accessibilityTraits  = UIAccessibilityTraitButton

        // 4.
        accessibleElements.append(elementForTapMe)

    }
}

1. Create UIAccessibilityElement for tapMe
2. Compute frame data on device's coordinates. Don't forget that frame's origin is the top/left corner for UIKit
3. Set data for UIAccessibilityElement
4. Add this UIAccessibilityElement to list of all accessible elements in scene.

Now tapMe is accessible from UI testing perspective.

References

• Session 406, UI Testing in Xcode, WWDC 2015

• eyes off eyes on — Voiceover accessibility in SpriteKit

• How do I support VoiceOver in a SpriteKit game? | Apple Developer Forums

• swift - Accessibility (Voice Over) with Sprite Kit - Stack Overflow