OCUnit tests failing, but Xcode Log Navigator says No Issues

Question

Im having a very sneaky issue in Xcode and its "Log Navigator".

I've recreated the issue like so; I have a very simple OCUnit test:

-(void)testSimpleFailure{

    STFail(@"Please fail!");
}

I clean my build, I build my test scheme, and run my tests.

In the output console window I see all my test execute, including the one above. The console displays the test actually ran AND failed:

##teamcity[testFailed name='-|[SimpleTestClassTests.testSimpleFailure|]' message='Please fail!' details='']

HOWEVER, Xcode displays the "Tests Succeeded overlay" and the "Log Navigator" on the left says there are no issues and all is green.

Has anyone else had a similar issue? Does anyone know how to resolve this?

I would very much like to not get a false positive before I commit.

Edited to clarify my question.

Answer 1

I'm running my application-tests and unit-tests on a device. I'm getting the same results as you - tests are failing but the Log Navigator says everything passes. Not only that, I can't set breakpoints in my code.

The solution for me was to "Edit Scheme...", chose the "Test" scheme, and change my Debugger from "LLDB" to "GDB". I'm using Xcode 4.3.2, with my compiler set for all targets to LLVM 3.1

After changing my debugger for the Test scheme to GDB, The Log Navigator reports the test failures, and I can set breakpoints and debug as well while the tests are running.

link: search for "Unit tests crash before executing any code" (LLDB bug) https://github.com/kstenerud/iOS-Universal-Framework

Answer 2

Ok, after about 3 developers over 5 days looking at this problem, we have found a solution:

rm -rf Library/Application\ Support/iPhone\ Simulator

Or, alternatively, if blowing away an entire directory tree isn't your thing:

Go to your iOS Simulator application. Choose iOS Simulator from the top menu and choose "Reset Content and Settings..."

Apparently something gets corrupted between the simulator and Xcode, so Xcode cannot hook to the simulator to watch the tests. Xcode believes no tests ran, therefore the tests pass.

Exactly what causes this, we aren't sure, but after 5 grueling days and 1 solution later, we don't care because we're heading to the nearest bar.

Cheers.