xcodecontinuous-integrationxctestxcodebuildgithub-actions

Xcode test failing does not failed Github action pipeline


I have a Github Action pipeline:

name: default

on: [push]

jobs:
  build:

    runs-on: macOS-latest

    steps:
    - uses: actions/checkout@v1
    - name: CocoaPod Install
      run: pod install
    - name: Force xcode 11
      run: sudo xcode-select -switch /Applications/Xcode_11.1.app
    - name: Test
      run: ./pipelines.sh test
    - name: Save report
      uses: actions/upload-artifact@v1
      with:
        name: test_report
        path: ./Test

And a script Shell:

function test
{
    xcodebuild \
       -workspace MyApp.xcworkspace \
       -scheme DEBUG\ -\ MyApp \
       -destination 'platform=iOS Simulator,name=iPhone 11' \
       test
}

My problem is when i run my pipeline with failing test, the pipeline is marked as PASSED, which is a problem...

I have also check with fastlane, failing test does not fail the pipeline.

How can I make my pipeline as FAIL when test does not pass?

Cf screenshot for fastlane: enter image description here


Solution

  • You need to return a non-zero value to fail the step. Try adding this to the xcodebuild command.

    xcodebuild -... || exit 1
    

    There are some other solutions besides this at the following question. How to get the return value of xcodebuild?

    Update: Based on your comments that you would like steps afterwards to complete you can do the following.

    Change your script to set an output containing the result of the xcodebuild command.

        xcodebuild \
           -workspace MyApp.xcworkspace \
           -scheme DEBUG\ -\ MyApp \
           -destination 'platform=iOS Simulator,name=iPhone 11' \
           test
        echo "::set-output name=result::$?"
    

    Add an id to the step where this script is executed.

        - name: Test
          id: xcodebuild
          run: ./pipelines.sh test
    

    At the end of your workflow you can check if the tests did not pass and fail the workflow.

          - name: Check Tests Passed
            if: steps.xcodebuild.outputs.result != 0
            run: exit 1