iosswiftxctestxcuitesttestrail

XCUITest integration with TestRail


Currently working on integrating my UITest-run results into TestRail, so after every test run it marks my tests as Pass\Fail in testrail.

The idea I have is:

  1. In CI create a "pre-build" script that will create a test run in testrail.
  2. During execution of automation, in test tearDown() get the result of a test(if a test failed or not), save it all into json file. - here is the first question, how can I get if a test failed?
  3. Once all tests are done, run a "post-build" script to get updated json file and send request to test rail(which will mark pass\fail tests)

Anyone who already worked on this, does it sound right for you? Any advice?

example of the test:

import XCTest

class MyUITests: XCTestCase {

    override func setUp() {
        super.setUp()
        continueAfterFailure = false
        appEntry.app.launch()
        dismissSystemAlerts()
    }

    override func tearDown() {
        super.tearDown()
    }

    func test_Elements() {
        // MARK: Sample test
        // my test actions are here
    }
}

Solution

  • This is how I implemented. First I have pre-build script in my CI that will create new Test Run in TestRail. And then UITestObserver will send APIs to TR to update statuses.

    new class that I've added:

    import Foundation
    import XCTest
    
    class UITestObserver: NSObject, XCTestObservation {
        // Handle Test Case Failure
        public func testCase(_ testCase: XCTestCase,
                             didFailWithDescription description: String,
                             inFile filePath: String?,
                             atLine lineNumber: Int) {
            var testCaseId: [String] = []
            if let MyTestCaseID = testCase as? BaseTest { testCaseId = MyTestCaseID.inegrateTestRailId() }
            if testCaseId != ["NA"] {
                postTestRailAddResultAPI(for: testCase, testCaseId: testCaseId, description: description)
            }
        }
    
        // Handle Test Case Pass
        public func testCaseDidFinish(_ testCase: XCTestCase) {
            let testRun = testCase.testRun!
            let verb = testRun.hasSucceeded
            var testCaseId: [String] = []
            if let MyTestCaseID = testCase as? BaseTest { testCaseId = MyTestCaseID.inegrateTestRailId() }
            if verb == true && testCaseId != ["NA"] {
                postTestRailAddResultAPI(for: testCase, testCaseId: testCaseId, description: "PASS")
        }
    }
    

    In BaseTest setUp added this line:

    XCTestObservationCenter.shared.addTestObserver(UITestObserver())
    

    And implemented function postTestRailAddResultAPI that sends actual requests to update status. All my tests now have testCaseId which stores value of TestRail TestCase number, and this is how it knows which TCs to update.