Shared XCTest unit tests for different implementations of interface

Question

I have two or more implementations of some interface (protocol):

protocol Interface {
    func methodOne()
    func methodTwo()
}

I want to test each implementation and I don't want to duplicate code. I have couple of options, but none of them satisfies me.

First one is to create test case for ImplementationA and subclass it to get test case for ImplementationB:

class ImplementationATests: XCTestCase {

    var implToTest: Interface!

    override func setUp() {
        super.setUp()
        implToTest = ImplementationA()
    }

    func testMethodOne() {
        ...
    }

    func testMethodTwo() {
        ...
    }
}


class ImplementationBTests: ImplementationATests {

    override func setUp() {
        super.setUp()
        implToTest = ImplementationB()
    }
}

One of the drawbacks of this method is that I can't have tests which apply only for ImplementationA. (e.g. to test some helper method specific to that implementation)

Second option I came up with is creating shared subclass for test cases:

class InterfaceTests: XCTestCase {

    var implToTest: Interface!

    func testMethodOne() {
        ...
    }

    func testMethodTwo() {
        ...
    }
}

But here those tests will be also executed, and they will fail, because no implementation is assigned to implToTest. Of course I can assign some implementation to it, but then I will end with two test cases for the same implementation. The best option would be to somehow disable InterfaceTests test case and run only its subclasses. Is it possible?

Third idea I got may seem tricky, but it would satisfy all my needs. Unfortunately it doesn't work. I decided to create InterfaceTestable protocol:

protocol InterfaceTestable {
    var implToTest: Interface! { get set }
}

and make extension to it with all shared tests:

extension InterfaceTestable {

    func testMethodOne() {
        ...
    }

    func testMethodTwo() {
        ...
    }
}

and then create test cases for each implementation:

class ImplementationATests: XCTestCase, InterfaceTestable {

    var implToTest: Interface!

    override func setUp() {
        super.setUp()
        implToTest = ImplementationA()
    }

    // some tests which only apply to ImplementationA
}


class ImplementationBTests: XCTestCase, InterfaceTestable {

    var implToTest: Interface!       

    override func setUp() {
        super.setUp()
        implToTest = ImplementationB()
    }

    // some tests which only apply to ImplementationB
}

Those test cases compile but Xcode doesn't see tests declared in InterfaceTestable extension.

Is there any other way to have shared tests for different implementations?

Answer 1

The way I've done this before is with a shared base class. Make implToTest nillable. In the base class, if an implementation is not provided, simply return out of the test in a guard clause.

It's a little annoying that the test run includes reports of the base class tests when it's not doing anything. But that's a small annoyance. The test subclasses will provide useful feedback.

Answer 2

Building on top of ithron's solution, if you carefully craft your defaultTestSuite, you can remove the need for each subclass to re-override it.

class InterfaceTests: XCTestCase {
    override class var defaultTestSuite: XCTestSuite {
        // When subclasses inherit this property, they'll fail this check, and hit the `else`.
        // At which point, they'll inherit the full test suite that generated for them.
        if self == AbstractTest.self {
            return XCTestSuite(name: "Empty suite for AbstractSpec")
        } else {
            return super.defaultTestSuite
        }
    }
}

Of course, the same limitation applies: this won't hide the empty test suite from the Xcode test navigator.

Generalizing this into a AbstractTestCase class

I would go a step further an make a base class AbstractTestCase: XCTestCase to store this defaultTestSuite trick, from which all your other abstract classes can inherit.

For completeness, to make it truly abstract you'd also want to override all the XCTestCase initializers to make them error out if an attempt is made to instantiate your abstract classes. On Apple's platforms, there's 3 initializers to override:

1. -[XCTestCase init]
2. -[XCTestCase initWithSelector:]
3. -[XCTestCase initWithInvocation:]

Unfortunately, this can't be done from Swift because of that last initializer, which uses NSInvocation. NSInvocation isn't available with Swift (it isn't compatible with Swift's ARC). So you need to implement this in Objective C. Here's my stab at it:

AbstractTestCase.h

#import <XCTest/XCTest.h>

@interface AbstractTestCase : XCTestCase

@end

AbstractTestCase.m

#import "AbstractTestCase.h"

@implementation AbstractTestCase

+ (XCTestSuite *)defaultTestSuite {
    if (self == [AbstractTestCase class]) {
        return [[XCTestSuite alloc] initWithName: @"Empty suite for AbstractTestCase"];
    } else {
        return [super defaultTestSuite];
    }
}

- (instancetype)init {
    self = [super init];
    NSAssert(![self isMemberOfClass:[AbstractTestCase class]], @"Do not instantiate this abstract class!");
    return self;
}

- (instancetype)initWithSelector:(SEL)selector {
    self = [super initWithSelector:selector];
    NSAssert(![self isMemberOfClass:[AbstractTestCase class]], @"Do not instantiate this abstract class!");
    return self;
}

- (instancetype)initWithInvocation:(NSInvocation *)invocation {
    self = [super initWithInvocation:invocation];
    NSAssert(![self isMemberOfClass:[AbstractTestCase class]], @"Do not instantiate this abstract class!");
    return self;
}

@end

Usage

You can then just use this as the superclass of your abstract test, e.g.

class InterfaceTests: AbstractTestCase {
  var implToTest: Interface!

  func testSharedTest() {

  }
}

Answer 3

I came across the same problem and solved it using your second option. However, I found a way to prevent the test cases from the base class from running:

Override the defaultTestSuite() class method in your base class to return an empty XCTestSuite:

class InterfaceTests: XCTestCase {

  var implToTest: Interface!

  override class func defaultTestSuite() -> XCTestSuite {
    return XCTestSuite(name: "InterfaceTests Excluded")
  }
}

With this no tests from InterfaceTests are run. Unfortunately also no tests of ImplementationATests either. By overriding defaultTestSuite() in ImplementationATests this can be solved:

class ImplementationATests : XCTestCase {

  override func setUp() {
    super.setUp()
    implToTest = ImplementationA()
  }

  override class func defaultTestSuite() -> XCTestSuite {
    return XCTestSuite(forTestCaseClass: ImplementationATests.self)
  } 
}

Now the test suite of ImplementationATests will run all test from InterfaceTests, but no tests from InterfaceTests are run directly, without setting implToTest.