Error with XCTestExpectation: API violation - multiple calls made to -[XCTestExpectation fulfill]

Question

I'm using XCTestExpectations in Xcode 6 (Beta 5) for asynchronous testing. All my asynchronous tests pass individually every time I run them. However, when I try to run my entire suite, some tests do not pass, and the app crashes.

The error I get is says API violation - multiple calls made to -[XCTestExpectation fulfill]. Indeed, this is not true within a single method; my general format for my tests is shown below:

- (void) someTest {
    /* Declare Expectation */
    XCTestExpectation *expectation = [self expectationWithDescription:@"My Expectation"];
    [MyClass loginOnServerWithEmail:@"[email protected]" andPassword:@"asdfasdf" onSuccess:^void(User *user) {
        /* Make some assertions here about the object that was given. */

        /* Fulfill the expectation */
        [expectation fulfill];
    }];

    [self waitForExpectationsWithTimeout:5.0 handler:^(NSError *error) {
        /* Error handling here */
    }];
}

Again, these tests do pass when run individually, and they are actually making network requests (working exactly as intended), but together, the collection of tests fail to run.

I was able to have a look at this post here, but was unable to get the solution to work for me.

Additionally, I'm running OSX Mavericks and using Xcode 6 (Beta 5).

Answer 1

I don't think using __weak or __block is a good approach. I have written many unit tests using XCTestExpectation for awhile and never had this problem until now. I finally found out that real cause of the problem which potentially may cause bugs in my app. The root cause of my problem is that startAsynchronousTaskWithDuration calls the completionHandler multiple time. After I fix it the API violation went away!

[self startAsynchronousTaskWithDuration:4 completionHandler:^(id result, NSError *error) {
    XCTAssertNotNil(result);
    XCTAssertNil(error);
    [expectation fulfill];
}];

Although it took me a few hours to fix my unit tests but I came to appreciate the API violation error which will help me avoid future runtime problem in my app.

Answer 2

Error: API violation - multiple calls made to -[XCTestExpectation fulfill]

I got the same error when set expectation.expectedFulfillmentCount.

Solution:

expectation.assertForOverFulfill = false

If expectation.assertForOverFulfill = true and fulfill() is called when it is already fulfilled then it throws an exception

Answer 3

Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'API violation - multiple calls made to -[XCTestExpectation fulfill] for whatever.'

I get the error above with the following code:

func testMultipleWaits() {
    let exp = expectation(description: "whatever")
    for _ in 0...10 {
        DispatchQueue.main.async {
            exp.fulfill()
        }
        wait(for: [exp], timeout: 1)
    }
}

Basically a given expectation can be fulfilled once, it can also be waited once.

Meaning the following change would still not fix it, because then you've still waited multiple times for it. It would give you the following error.

failed: caught "NSInternalInconsistencyException", "API violation - expectations can only be waited on once, whatever have already been waited on"

func testMultipleWaits() {
    let exp = expectation(description: "whatever")
    for i in 0...10 {
        DispatchQueue.main.async {
            if i == 6 {
                exp.fulfill()
            }
        }
        wait(for: [exp], timeout: 1)
    }
}

What makes it confusing is that while the above change still crashes, the test does get finished and gives you a failure of "Asynchronous wait failed: Exceeded timeout of 1 seconds, with unfulfilled expectations: "whatever".'

It's misleading because you may spend time fixing the test, all while you should be spending time fixing the crash which is the root cause

---

The fix here is set the expectation inside the for-loop. This way the expectation is fulfilled and waited against once per expectation.

func testMultipleWaits() {        
    for _ in 0...10 {
        let exp = expectation(description: "whatever")
        DispatchQueue.main.async {
            exp.fulfill()
        }
        wait(for: [exp], timeout: 1)
    }
}

Answer 4

Had the same issue on Xcode 12.5 / Swift 5.

Check if you're not overriding setUp and tearDown 'class' methods.

It should be:

override func setUp()
override func tearDown()

Instead of:

override class func setUp()
override class func tearDown()