azure-devopsazure-pipelinescode-coveragesonarcloud

In azure Devops, Run on unit Test on Multi-Agent, how to collect the code coverage file and Publish into Artifact


In azure Devops, Run Unit Test(VSTest) on Multi-Agent, how to collect the code coverage file and Publish into Artifact. In pipeline show only test result .trx File and unable to view .coverage File.

YAML Template File:

# Unit Test Execution      
- job: UnitTests_Execution
  displayName : 'Unit Tests execution'
  dependsOn: 'UnitTest_Build'
  condition: and(succeeded(), ${{ and(eq(parameters.RunCondition, 'True'), eq(parameters.RunUnitTest, 'True')) }})

  variables:
    BuildPlatform: 'any cpu'
    BuildConfiguration: 'release'
  
  pool:    
    name: 'Self-Hosted Agent'
  strategy:
    parallel: 8
      
  steps:
    - checkout: none
    
    - task: DownloadPipelineArtifact@2
      displayName: 'Download Build Artifacts'
      inputs:
        artifactName: unitTests
        targetPath: '$(System.ArtifactsDirectory)/unitTests'
    
    - task: VSTest@2
      displayName: 'Unit Tests'
      inputs:
        testSelector: 'testAssemblies'
        testAssemblyVer2: |
          **\*test*.dll
          !**\*TestAdapter.dll
          !**\obj\**
        searchFolder: '$(System.ArtifactsDirectory)/unitTests'
        resultsFolder: '$(System.ArtifactsDirectory)\TestResults\'
        runOnlyImpactedTests: false
        vsTestVersion: '${{ parameters.VsVersion }}'
        # runInParallel: true
        codeCoverageEnabled: true
        testRunTitle: 'Unit Testing service layer'
        platform: '$(BuildPlatform)'
        configuration: '$(BuildConfiguration)'
        rerunFailedTests: true

    - task: PublishPipelineArtifact@1 
      displayName: 'Publish Artifact: Code Coverage'
      inputs:
        targetPath: '$(build.ArtifactStagingDirectory)/TestResults'
        artifact: CodeCoverageResult

Using Multi-Agent, Am tried to collect the code covearge File and Merge into single File and publish into Sonar Cloud Dashboard

Excepted: In pipeline log only show the Test result .trx File and not show .coverage File. But in Single Agent it was working am excepting to work on multi-Agent concept.

enter image description hereenter image description here


Solution

  • I can reproduce the same issue when using the multiple-agents jobs.

    enter image description here

    It seems that the Code Coverage option doesn't support in vstest task when using multiple-agents jobs.

    The Pipeline runs will point to the same Test Run link and not execute the real test method.

    To solve this issue, you can use the parameters to loop the jobs to run the multiple jobs.

    Here is an example:

    parameters:
    - name: runtimes
      type: object
      default: [1,2,3,4,5,6,7,8]
    
    
    jobs:
    - ${{ each runtime in parameters.runtimes }}:
    
      - job: UnitTests_Execution_${{ runtime }}
        displayName : 'Unit Tests execution ${{ runtime }}'
        dependsOn: 'UnitTest_Build'
        condition: and(succeeded(), ${{ and(eq(parameters.RunCondition, 'True'), eq(parameters.RunUnitTest, 'True')) }})
    
        variables:
          BuildPlatform: 'any cpu'
          BuildConfiguration: 'release'
        
        pool:    
          name: 'Self-Hosted Agent'
    
            
        steps:
          - checkout: none
          
          - task: DownloadPipelineArtifact@2
            displayName: 'Download Build Artifacts'
            inputs:
              artifactName: unitTests
              targetPath: '$(System.ArtifactsDirectory)/unitTests'
          
          - task: VSTest@2
            displayName: 'Unit Tests'
            inputs:
              testSelector: 'testAssemblies'
              testAssemblyVer2: |
                **\*test*.dll
                !**\*TestAdapter.dll
                !**\obj\**
              searchFolder: '$(System.ArtifactsDirectory)/unitTests'
              resultsFolder: '$(System.ArtifactsDirectory)\TestResults\'
              runOnlyImpactedTests: false
              vsTestVersion: '${{ parameters.VsVersion }}'
              # runInParallel: true
              codeCoverageEnabled: true
              testRunTitle: 'Unit Testing service layer'
              platform: '$(BuildPlatform)'
              configuration: '$(BuildConfiguration)'
              rerunFailedTests: true
    
          - task: PublishPipelineArtifact@1 
            displayName: 'Publish Artifact: Code Coverage'
            inputs:
              targetPath: '$(build.ArtifactStagingDirectory)/TestResults'
              artifact: CodeCoverageResult
    

    In this case, it will run 8 parallel jobs too. And it can show the expected behavior.

    Here is the result: .trx file and .coverage file

    enter image description here