XCTAssertTrue doesn't stop routine

Question

Xcode doesn't terminate a test routine at failed assertions. Is this correct? I fail to understand the reason behind this and I'd like it to behave like assert and have it terminate the program. With the following test, it will print "still running". Is this intended?

- (void)testTest
{
    XCTAssertTrue(false, @"boo");
    NSLog(@"still running");
}

I don't see how this would be useful, because often subsequent code would crash when pre-conditions aren't met:

- (void)testTwoVectors
{
    XCTAssertTrue(vec1.size() == vec2.size(), @"vector size mismatch");

    for (int i=0; i<vec1.size(); i++) {
        XCTAssertTrue(vec1[i] == vec2[i]);
    }
}

Answer 1

Yes, it is intended. That's how unit tests work. Failing a test doesn't terminate the testing; it simply fails the test (and reports it as such). That's valuable because you don't want to lose the knowledge of whether your other tests pass or fail merely because one test fails.

If (as you say in your addition) the test method then proceeds to throw an exception, well then it throws an exception - and you know why. But the exception is caught, so what's the problem? The other test methods still run, so your results are still just what you would expect: one method fails its test and then stops, the other tests do whatever they do. You will then see something like this in the log:

Executed 7 tests, with 2 failures (1 unexpected) in 0.045 (0.045) seconds

The first failure is the XCTAssert. The second is the exception.

Answer 2

you can change this behavior of XCTAssert<XX>.
In setup method change value self.continueAfterFailure to NO.

IMO stopping the test after test assertion failure is better behavior (prevents crashes what lead to not running other important tests). If test needs continuation after a failure this means that test case is simply to long an should be split.