How do you debug a unit test in Xcode 3?

Question

I followed Apple's instructions to set up Unit Testing in my project. I followed the directions for making them dependent, so the tests run with every build of my main project. This works, and when my tests pass, the application runs; when they don't, I get build errors on the lines of the unit tests that failed.

I would like, however, to be able to step through my application code when the tests are failing, but can't get Xcode (3.2.5) configured properly. The project is a Mac project, not iOS.

I tried the instructions here and here, but execution never stopped at the breakpoints I set, neither in the the unit test code or in my application code. After following the first set of instructions, the breakpoints I set turned yellow with blue outlines, and I don't know what that meant, either. What do I need to do to step through my tests?

Update

I found another page attempting to solve this problem (here) by adding arguments and environment variables to my main executable, but again, those didn't cause execution to stop at my breakpoints. I also noticed that my test's (copious) log statements aren't showing up in my Debugger Console, either.

I also found out that the yellow breakpoints mean the code there can't be found at runtime. These are in my test case class, so that definitely seems to explain why those aren't firing.

Answer 1

I usually have no problem debugging my OCTest tests with Xcode 3.2, including halting at breakpoints.

The basic idea is to tell gdb to launch otest with your bundle as argument. You will do that by adding /Developer/Tools/otest as a custom executable to your Xcode project, then your OCTest bundle name as the sole argument (choose "Edit Active Executable otest" in the Project menu, and in the second tab add a line with Foo.octest in the top box to debug the Foo test).

Now if you hit the debug button it will start debugging your test bundle right now, and will stop at the declared breakpoints (if you hit build and debug it might not start if the test does not pass). Also note that you will maybe have to set an environment variable to YES to disable garbage collection (in the bottom box of the same "Arguments" tab), otest will tell you exactly which if you need.

If you did everything above and are still unable do step into your test code, this might either be because it is compiled with debug symbol generation turned off - check your debug build settings, but most likely because the test code is not recompiled at all. I say most likely, since your logs are not showing in the console, NSLog should write in the Xcode console. Clean your build and bin folders manually, sometimes when you change a path or a name you end up loading obsolete code. You might also want to check that the file has not jumped out of the test target (is the failing test in the same not-logging file ?).

Answer 2

I have a variant of what Hiedi Utley posted at http://hiediutley.wordpress.com/2011/03/08/xcode3-debugging-ios-unit-tests/. What I did not like about it is the duplication of unit test bundle targets, one that contained a Run Script phase to execute the unit tests after building, and another that did not run the unit tests. I had noticed in the Get Info pane for the Run Script phase the toggle "Run script only when installing" and thought that that would be a way to toggle between running the unit tests in normal mode and running them in the debugger.

As per Hiedi's instructions, create a new Executable item, say LogicTestsGDB. Configure it like so:

General Tab:

• Path: Developer/usr/bin/otest (No leading /)
• Path Type: Relative to Current SDK
• Set the working directory to: Build Products directory

Arguments Tab:

• Arguments: Your UnitTest Bundle (eg. LogicTests.octest)
• Variables to be set in the environment
  • DYLD_LIBRARY_PATH ... : ${BUILD_PRODUCTS_DIR}:${DYLD_LIBRARY_PATH}
  • DYLD_FRAMEWORK_PATH . : ${SDKROOT}/Developer/Library/Frameworks
  • DYLD_ROOT_PATH ...... : ${SDKROOT}
  • IPHONE_SIMULATOR_ROOT : ${SDKROOT}
  • OBJC_DISABLE_GC : YES
  • DYLD_NEW_LOCAL_SHARED_REGIONS : YES
  • DYLD_NO_FIX_PREBINDING : YES
  • CFFIXED_USER_HOME : ${HOME}/Library/Application Support/iPhone Simulator/

Done. Now to debug your unit tests,

1. Expose the bundle's build phases and double-click on the Run Script phase.
2. Check the Run script only when installing option
3. Set the Active Target to be the unit test bundle (eg. LogicTests.octest).
4. Set the Active Executable to be the new executable created (eg. LogicTestsGDB)
5. Click Build and Debug

To run normally, executing unit tests as part of the build phase of the application:

1. Expose the bundle's build phases and double-click on the Run Script phase.
2. Uncheck the Run script only when installing option
3. Set the Active Target to be the application being built
4. Set the Active Executable to be the application being built

To automate the steps above, I created a simple AppleScript script that toggles between the two states:

property kApplicationName : "MyApp" -- name of the normal application to build
property kUnitTestName : "LogicTests" -- name of the bundle target to debug
property kUnitTestRunner : "LogicTestGDB" -- name of the executable to use when debugging the unit test bundle

tell application "Xcode"
    tell the active project document
    set theTarget to first target whose name is kUnitTestName
        set thePhase to first run script phase of theTarget
        if name of active target is kApplicationName then
            set active target to theTarget
            set theExecutable to first executable whose name is kUnitTestRunner
            set active executable to theExecutable
            set run only when installing of thePhase to true
        else
            set theTarget to first target whose name is kApplicationName
            set active target to theTarget
            set theExecutable to first executable whose name is kApplicationName
            set active executable to theExecutable
            set run only when installing of thePhase to false
        end if
        return "Targeting " & (name of active executable)
    end tell
end tell