XCTest passes when it should fail using expectations

Question

I am testing a method that runs in background and executes a code block when it finishes. I am using expectations to handle the asynchronous execution of the tests. I wrote simple a test that shows the behaviour:

- (void) backgroundMethodWithCallback: (void(^)(void)) callback {
    dispatch_queue_t backgroundQueue;
    backgroundQueue = dispatch_queue_create("background.queue", NULL);
    dispatch_async(backgroundQueue, ^(void) {
        callback();
    });
}

- (void) testMethodWithCallback {
    XCTestExpectation *expectation = [self expectationWithDescription:@"Add collection bundle"];
    [self backgroundMethodWithCallback:^{
        [expectation fulfill];

        usleep(50);
        XCTFail(@"fail test");
    }];
    [self waitForExpectationsWithTimeout: 2 handler:^(NSError *error) {
        if (error != nil) {
            XCTFail(@"timeout");
        }
    }];
}

The XCTFail(@"fail test"); line should fail for this test but the test is passing.

I also noticed that this only happens when the code ran on the callback takes an amount of time (in my case, I was checking some files on the file system). This is why the usleep(50); line is necessary to reproduce the case.

Answer 1

The expectation must be fulfilled after all the test checks. Moving the line to the end of the callback block is enough to make the test fail:

- (void) testMethodWithCallback {
    XCTestExpectation *expectation = [self expectationWithDescription:@"Add collection bundle"];
    [self backgroundMethodWithCallback:^{

        usleep(50);
        XCTFail(@"fail test");
        [expectation fulfill];
    }];
    [self waitForExpectationsWithTimeout: 2 handler:^(NSError *error) {
        if (error != nil) {
            XCTFail(@"timeout");
        }
    }];
}

I did not find explicit documentation about this but in the apple developer guides, the fulfill message is sent at the end of the block and it makes a lot of sense.

Note: I first found an example in swift where the fulfill method is called at the start of the callback. What I don't know is if the example is not correct or there is a difference with Objective-C.