Where do Xcode Bots put their results, so I can parse them?

Question

Our dev team has always used Jenkins for our iOS builds, and used Philips Hue lights to inform the team when the build is Building(Yellow), Successful(Green), Failed(Red).

Now we have moved to Xcode CI and Bots, and I do not know when any unit tests fail. We don't even know if the build phase failed.

On Xcode Bots CI you get this "bigscreen" feature: In Apple's "Manage and Monitor Bots from a Web Browser" Docs, you can see that it has all sorts of states that could key up a hue light.

I really don't want to hack something up and parse an HTML page. Although fun, the work does not last long if Apple updates their HTML markup.

Is there a parseable file that is produced when a Xcode bot finishes its integration?

I'd love to get the Hue showing:
* Blue for Analysis warnings
* Orange for Build Warnings
* Red for Build Errors
* Yellow for build running

Answer 1

I'd like to share the team's solution. We found the place where the Bot Results are stored, we parse it using bash, and send messages to the HUE light via curl system calls. We call the script in a scheme pre and post Build scripts.

We parse the bot's result plist at:

/Library/Server/Xcode/Data/BotRuns/Latest/output/xcodebuild_result.bundle/Info.plist

There you can find all sorts of cool data to use!:

<dict>
    <key>AnalyzerWarningCount</key>
    <integer>0</integer>
    <key>AnalyzerWarningSummaries</key>
    <array/>
    <key>ErrorCount</key>
    <integer>0</integer>
    <key>ErrorSummaries</key>
    <array/>
    <key>LogIdentifier</key>
    <string>705bffcb-7453-49ba-882f-80e1218b59cf</string>
    <key>LogPath</key>
    <string>1_Test/action.xcactivitylog</string>
    <key>Status</key>
    <string>IDEActionResultStatus_Succeeded</string>
    <key>TestFailureSummaries</key>
    <array/>
    <key>TestSummaryIdentifier</key>
    <string>a1554874-4d40-4e94-ae89-a73184ec97a9</string>
    <key>TestSummaryPath</key>
    <string>1_Test/action_TestSummaries.plist</string>
    <key>TestsCount</key>
    <integer>185</integer>
    <key>TestsFailedCount</key>
    <integer>0</integer>
    <key>WarningCount</key>
    <integer>0</integer>
    <key>WarningSummaries</key>
    <array/>
<dict>

• AnalyzerWarningCount
• ErrorCount
• WarningCount
• TestsFailedCount

Oh bash, my sometimes lover, come save the day again.

Also notice the use of the Plist Buddy for parsing Xcode's XML property list files. The primo choice for getting information in and out of plist files.

    #!/bin/bash
    #
    #   By Phil
    #
    exec > /tmp/my_log_file.txt 2>&1
    TEST_RESULT_PLIST="/Library/Server/Xcode/Data/BotRuns/Latest/output/xcodebuild_result.bundle/Info.plist"

    hue_light_green=false

    echo "testResultParse_OwlHue"

    #If not bot, return
    if [ "$(whoami)" != "_teamsserver" ]; then
        echo "$(whoami) - Not a bot!";
        exit 1
    fi

    #1 If file not found ERROR
    if [ ! -f $TEST_RESULT_PLIST ]; then
        curl -X PUT -d "{\"on\":true,\"bri\":32,\"effect\":\"none\",\"hue\":150,\"sat\":255,\"alert\":\"lselect\"}" ipaddress/api/testestest/lights/3/state
        echo "Test Result Plist not Found";
        exit 1
    fi

    #2 AnalyzerWarningCount BLUE
    AnalyzerWarningCount=$(/usr/libexec/PlistBuddy -c "Print :AnalyzerWarningCount" "${TEST_RESULT_PLIST}")
    if [ $AnalyzerWarningCount != 0 ]; then
        echo "AnalyzerWarningCount";
        curl -X PUT -d "{\"on\":true,\"bri\":32,\"xy\":[0.16, 0.1],\"hue\":15815,\"sat\":255,\"effect\":\"none\",\"alert\":\"lselect\"}" ipaddress/api/testestest/lights/3/state
    fi

    #3 WarningCount
    WarningCount=$(/usr/libexec/PlistBuddy -c "Print :WarningCount" "${TEST_RESULT_PLIST}")
    if [ $WarningCount != 0 ]; then
        curl -X PUT -d "{\"on\":true,\"bri\":32,\"xy\":[0.58, 0.41],\"hue\":15815,\"sat\":255,\"effect\":\"none\",\"alert\":\"lselect\"}" ipaddress/api/testestest/lights/3/state
        echo "WarningCount";
    fi

    #4 ErrorCount || TestsFailedCount ERROR
    ErrorCount=$(/usr/libexec/PlistBuddy -c "Print :ErrorCount" "${TEST_RESULT_PLIST}")
    if [ $ErrorCount != 0 ]; then
        curl -X PUT -d "{\"on\":true,\"bri\":32,\"effect\":\"none\",\"hue\":150,\"sat\":255,\"alert\":\"lselect\"}" ipaddress/api/testestest/lights/3/state
        echo "ErrorCount";
        exit 1
    fi

    #5 TestsFailedCount ERROR
    ErrorCount=$(/usr/libexec/PlistBuddy -c "Print :ErrorCount" "${TEST_RESULT_PLIST}")
    if [ $TestsFailedCount != 0 ]; then
        curl -X PUT -d "{\"on\":true,\"bri\":32,\"effect\":\"none\",\"hue\":150,\"sat\":255,\"alert\":\"lselect\"}" ipaddress/api/testestest/lights/3/state
        echo "TestsFailedCount";
        exit 1
    fi

    #6 None of the above.  SUCCESS
    if [ "$hue_light_green" = true ] ; then
        echo "SUCCESS";
        curl -X PUT -d "{\"on\":true,\"bri\":32,\"effect\":\"none\",\"hue\":25500,\"sat\":255,\"alert\":\"lselect\"}" ipaddress/api/testestest/lights/3/state
    fi

• AnalyzerWarningCount Blue
• ErrorCount Red
• WarningCount Orange
• TestsFailedCount Red

Now when we get a count for any of the above, we get a blinking color change. For example, the following produces a bright blue from our hue:

Answer 2

The result path for OS X Server 4.0 seems to be:

/Library/Developer/XcodeServer/IntegrationAssets/Your_Bot/

• Archive.xcarchive.zip
• build.log
• buildService.log
• Your_Bot.ipa
• sourceControl.log
• xcodebuild_result.bundle.zip

The xcodebuild_result.bundle is a zip file now, I parse the build result from buildService.log instead:

Build results summary: {
analyzerWarningChange = 14;
analyzerWarningCount = 14;
errorChange = 0;
errorCount = 0;
improvedPerfTestCount = 0;
regressedPerfTestCount = 0;
testFailureChange = 0;
testFailureCount = 0;
testsChange = 0;
testsCount = 0;
warningChange = 20;
warningCount = 20;
}