Shared tests in XCTest test suites

Question

I have a class which can be configured to do 2 slightly different things. I want to test both paths. The class is a descendant of UIViewController and most of the configuration takes place in Interface Builder. i need to verify that both Storyboard Scenes and their Outlets are wired up in the same way, but also need to check for the difference in behavior.

I'd like to use shared XCTest suites for this purpose.

One is aimed for use with the left hand, one for the right. Both appear after another when using the app. The first one (right hand) triggers a segue to the other. The last one (left hand) should trigger a different segue. This is where it differs, for example.

Now I want to verify the segues with tests. I'd like to create a BothHandSharedTests suite which both view controller instance tests use to verify everything they have in common. However, the BothHandSharedTests class is treated as a self-containing test suite, which it clearly isn't.

I came up with these strategies:

• inherit from an abstract XCTest descendant, like described above (doesn't seem to be that easy),
• write a test auite for the common properties and use one of the two as the Object Under Test, and add two smaller suites for the differences.

How would you solve this problem?

Answer 1

Here's a solution in Swift:

class AbstractTests: XCTestCase {

    // Your tests here

    override func perform(_ run: XCTestRun) {
        if type(of: self) != AbstractTests.self {
            super.perform(run)
        }
    }
}

Answer 2

I don't have a conclusive answer, but here's what I ended up doing.

I first tried the subclassing route. In the parent test (the "AbstractTestCase") I implemented all the tests that would be executed by the the AbstractTestCase subclasses, but added a macro so they don't get run by the actual parent test:

#define DONT_RUN_TEST_IF_PARENT if ([[self className] isEqualToString:@"AbstractTestCase"]) { return; }

I then added this macro to the start of every test, like so:

- (void)testSomething
{
    DONT_RUN_TEST_IF_PARENT

    ... actual test code ...
}

This way, in the ConcreteTestCase classes which inherit from AbstractTestCase, all those tests would be shared and run automatically. You can override -setUp to perform the necessary class-specific set-up, of course.

However – This turned out to be a crappy solution for a couple reasons:

1. It confuses Xcodes testing UI. You don't really get to see a live representation of what's running, and tests sometimes don't show up as intended. This makes clicking through to debug test failures difficult or impossible.
2. It confuses XCTest itself – I found that tests would often get run even when I didn't ask them too (if I were just trying to run a single test) and the so the test output wouldn't be what I would expect.
3. Honestly it felt a little janky to have that macro – macros which redirect flow control are never really that good of an idea.

Instead, I'm now using a shared object, a TestCaseHelper, which is instantiated for each test class, and has has a protocol/delegate pattern common to all test cases. It's less DRY – most test cases are just duplicates of the others – but at least they are simple. This way, Xcode doesn't get confused, and debugging failures is still possible.

A better solution will likely have to come from Apple, unless you're interested in ditching your entire test suite for something else.