Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

XCUITest integration with TestRail

Currently working on integrating my UITest-run results into TestRail, so after every test run it marks my tests as Pass\Fail in testrail.

The idea I have is:

  1. In CI create a "pre-build" script that will create a test run in testrail.
  2. During execution of automation, in test tearDown() get the result of a test(if a test failed or not), save it all into json file. - here is the first question, how can I get if a test failed?
  3. Once all tests are done, run a "post-build" script to get updated json file and send request to test rail(which will mark pass\fail tests)

Anyone who already worked on this, does it sound right for you? Any advice?

example of the test:

import XCTest

class MyUITests: XCTestCase {

    override func setUp() {
        super.setUp()
        continueAfterFailure = false
        appEntry.app.launch()
        dismissSystemAlerts()
    }

    override func tearDown() {
        super.tearDown()
    }

    func test_Elements() {
        // MARK: Sample test
        // my test actions are here
    }
}
like image 637
Dimu4 Avatar asked Mar 06 '23 17:03

Dimu4


1 Answers

This is how I implemented. First I have pre-build script in my CI that will create new Test Run in TestRail. And then UITestObserver will send APIs to TR to update statuses.

new class that I've added:

import Foundation
import XCTest

class UITestObserver: NSObject, XCTestObservation {
    // Handle Test Case Failure
    public func testCase(_ testCase: XCTestCase,
                         didFailWithDescription description: String,
                         inFile filePath: String?,
                         atLine lineNumber: Int) {
        var testCaseId: [String] = []
        if let MyTestCaseID = testCase as? BaseTest { testCaseId = MyTestCaseID.inegrateTestRailId() }
        if testCaseId != ["NA"] {
            postTestRailAddResultAPI(for: testCase, testCaseId: testCaseId, description: description)
        }
    }

    // Handle Test Case Pass
    public func testCaseDidFinish(_ testCase: XCTestCase) {
        let testRun = testCase.testRun!
        let verb = testRun.hasSucceeded
        var testCaseId: [String] = []
        if let MyTestCaseID = testCase as? BaseTest { testCaseId = MyTestCaseID.inegrateTestRailId() }
        if verb == true && testCaseId != ["NA"] {
            postTestRailAddResultAPI(for: testCase, testCaseId: testCaseId, description: "PASS")
    }
}

In BaseTest setUp added this line:

XCTestObservationCenter.shared.addTestObserver(UITestObserver())

And implemented function postTestRailAddResultAPI that sends actual requests to update status. All my tests now have testCaseId which stores value of TestRail TestCase number, and this is how it knows which TCs to update.

like image 100
Dimu4 Avatar answered Mar 15 '23 15:03

Dimu4