azure-devopsautomated-testsdevopsxcuitestvisual-studio-app-center-test

How to run XCUITests in Microsoft AppCenter from Azure DevOps


I've posted issues in the respective Microsoft Github repositories, but given that they redirected me to Stackoverflow, I'm gonna re-post the question here.

My question is: How to run the automated XCUITests in Microsoft AppCenter from Azure DevOps?

I studied the documentation on how to manually build the app from the console and then upload it to AppCenter (which works). Now I would like to use the official Azure App Center Test task, which should do a similar thing. Unfortunately, these two documentations are significantly different and I have no idea what information I have to provide to that step in order to make it work.

The biggest difference I noticed, is that the AppCenter documentation uses "build-for-testing" and a DerivedData directory where it builds stuff that gets uploaded, whereas the AppCenterTest task requests an IPA, a build directory and a Test IPA path. How to obtain these artifacts?

I tried something like this:

- task: Xcode@5
  inputs:
    actions: 'clean build test'
    configuration: 'Debug'
    sdk: 'iphoneos'
    xcWorkspacePath: 'MyProject/MyProject.xcworkspace'
    scheme: 'MyProject'
    packageApp: true
    exportPath: '$(build.artifactStagingDirectory)/debug'
    signingOption: 'manual'
    signingIdentity: '$(APPLE_CERTIFICATE_SIGNING_IDENTITY)'
    provisioningProfileUuid: '$(APPLE_PROV_PROFILE_UUID)'
    destinationPlatformOption: 'iOS'
    destinationSimulators: 'iPhone 11'
    publishJUnitResults: true

- task: Xcode@5
  inputs:
    actions: 'build-for-testing'
    configuration: 'Debug'
    sdk: 'iphoneos'
    xcWorkspacePath: 'MyProject/MyProject.xcworkspace'
    scheme: 'MyProject'
    packageApp: false
    args: '-derivedDataPath $(build.artifactStagingDirectory)/DerivedData'

- task: AppCenterTest@1
  inputs:
    appFile: '$(build.artifactStagingDirectory)/DerivedData/Build/Products/Debug-iphoneos/MyProject.app'
    artifactsDirectory: '$(build.artifactStagingDirectory)/AppCenterTest'
    frameworkOption: 'xcuitest'
    xcUITestBuildDirectory: '$(ProjectDir)/Build/Products/Debug-iphoneos'
    xcUITestIpaFile: '$(build.artifactStagingDirectory)/DerivedData/Build/Products/Debug-iphoneos/MyProjectUITests-Runner.app'
    credentialsOption: 'serviceEndpoint'
    serverEndpoint: 'MyCustomer AppCenter Deployment'
    appSlug: 'MyCustomer/MyProject-iOS'
    devices: 'iphoneDevices'
    localeOption: 'en_US'
    skipWaitingForResults: true

but I'm getting an error similar to ##[error]Error: Cannot find any file based on /Users/runner/runners/2.163.1/work/1/a/DerivedData/Build/Products/Debug-iphoneos/MyProject.app

Does anyone have a full working example and could provide an example YAML file that:


Solution

  • We ended up with the following build script that actually works. Please note, that you can't combine steps arbitrarily, as xcode build-for-testing does not allow providing signing certificates and thus cannot be combined with packageApp: true

    # Xcode
    # Build, test, and archive an Xcode workspace on macOS.
    # Add steps that install certificates, test, sign, and distribute an app, save build artifacts, and more:
    # https://learn.microsoft.com/azure/devops/pipelines/languages/xcode
    
    trigger:
    - master
    
    pool:
      vmImage: 'macos-latest'
    
    steps:
    ########################################
    # Install development certificates     #
    ########################################
    - task: InstallAppleCertificate@2
      inputs:
        certSecureFile: 'MyCompany-distribution.p12'
        certPwd: $(P12password)
        keychain: 'temp'
    
    - task: InstallAppleProvisioningProfile@1
      inputs:
        provisioningProfileLocation: 'sourceRepository'
        provProfileSourceRepository: 'Signing/Distribution_Profile.mobileprovision'
    
    ########################################
    # Build App and test it locally        #
    ########################################
    - task: Xcode@5
      inputs:
        actions: 'clean build test'
        configuration: 'Debug'
        sdk: 'iphoneos'
        xcWorkspacePath: 'TestApp-iOS/TestApp-iOS.xcodeproj'
        scheme: 'TestApp-iOS'
        destinationPlatformOption: 'iOS'
        destinationSimulators: 'iPhone 11'
        publishJUnitResults: true
    
    ###############################################
    # Run automated UI Tests on physical devices  #
    ###############################################
    - task: Xcode@5
      inputs:
        actions: 'build-for-testing'
        configuration: 'Debug'
        sdk: 'iphoneos'
        xcWorkspacePath: 'TestApp-iOS/TestApp-iOS.xcodeproj'
        scheme: 'TestApp-iOS'
        packageApp: false
        args: '-derivedDataPath $(build.artifactStagingDirectory)/DerivedData'
    
    - task: Xcode@5
      inputs:
        actions: 'build'
        configuration: 'Release'
        sdk: 'iphoneos'
        xcWorkspacePath: 'TestApp-iOS/TestApp-iOS.xcodeproj'
        scheme: 'TestApp-iOS'
        packageApp: true
        exportPath: '$(build.artifactStagingDirectory)/release'
        signingOption: 'manual'
        signingIdentity: '$(APPLE_CERTIFICATE_SIGNING_IDENTITY)'
        provisioningProfileUuid: '$(APPLE_PROV_PROFILE_UUID)'
        destinationPlatformOption: 'iOS'
        destinationSimulators: 'iPhone 11'
    
    - task: AppCenterTest@1
      inputs:
        appFile: '$(Build.ArtifactStagingDirectory)/release/TestApp-iOS.ipa'
        artifactsDirectory: '$(Build.ArtifactStagingDirectory)/AppCenterTest'
        frameworkOption: 'xcuitest'
        xcUITestBuildDirectory: '$(build.artifactStagingDirectory)/DerivedData/Build/Products/Debug-iphoneos'
        credentialsOption: 'serviceEndpoint'
        serverEndpoint: 'MyCompany AppCenter Deployment'
        appSlug: 'MyCompany/Moments-iOS'
        devices: 'MyCompany/iphone11'
        localeOption: 'en_US'
        skipWaitingForResults: true
        showDebugOutput: true