unit-testingazure-devopsazure-pipelinescode-coverage

BuildQualityChecks task doesn't consider all the published test coverage reports


I have 3 unit testing projects for 3 different projects in the solution.

When I run the build pipeline, those unit test projects generate 3 results and when code coverage task runs, it only pick one report.

How can I merge all the reports to one and validate against?

In the pipeline results if I go to the Code Coverage tab, It has the correct total line coverage value. output published

but the code coverage tool shows a less number output of the pipeline

the pipeline steps are as follows

- task: DotNetCoreCLI@2
  displayName: dotnet test $(buildConfiguration)
  inputs:
    command: test
    publishTestResults: true
    projects: '$(unitTestsDirs)/**/*.csproj'
    arguments: '--configuration $(buildConfiguration) --no-build --collect:"XPlat Code Coverage" --settings "./coverage.runsettings" -- DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=cobertura'
    
- task: PublishCodeCoverageResults@1
  displayName: 'Publish code coverage report'
  inputs:
    codeCoverageTool: 'Cobertura'
    summaryFileLocation: '$(Agent.TempDirectory)/**/coverage.cobertura.xml'
    
- task: BuildQualityChecks@9
  displayName: 'Check build quality'
  inputs:
    checkCoverage: true
    coverageFailOption: fixed
    coverageType: lines
    coverageThreshold: $(CoverageThreashold)
    buildPlatform: $(BuildPlatform)

Bits from outputs of the tests are as follows

Results File: /azp/_work/_temp/_rancher-ubuntu-on-prem-agents-nhpsb_2024-11-18_15_05_23.trx
Attachments:
/azp/_work/_temp/d0caa73f-caee-48d1-b626-f49093a0fb14/_rancher-ubuntu-on-prem-agents-nhpsb_2024-11-18.15_05_23.cobertura.xml
/azp/_work/_temp/d0caa73f-caee-48d1-b626-f49093a0fb14/coverage.cobertura.xml

Results File: /azp/_work/_temp/_rancher-ubuntu-on-prem-agents-nhpsb_2024-11-18_15_05_30.trx
Attachments:
/azp/_work/_temp/1c8c6bf9-96be-405d-b0b5-b852eb474a1b/_rancher-ubuntu-on-prem-agents-nhpsb_2024-11-18.15_05_31.cobertura.xml
/azp/_work/_temp/1c8c6bf9-96be-405d-b0b5-b852eb474a1b/coverage.cobertura.xml

Results File: /azp/_work/_temp/_rancher-ubuntu-on-prem-agents-nhpsb_2024-11-18_15_05_35.trx
Attachments:
/azp/_work/_temp/6bb84b94-f726-44b0-94ce-3047f7af5c91/_rancher-ubuntu-on-prem-agents-nhpsb_2024-11-18.15_05_36.cobertura.xml
/azp/_work/_temp/6bb84b94-f726-44b0-94ce-3047f7af5c91/coverage.cobertura.xml

Async Command Start: Publish test results
Publishing test results to test run '911742'.
TestResults To Publish 55, Test run id:911742
Test results publishing 55, remaining: 0. Test run id: 911742
Publishing test results to test run '911744'.
TestResults To Publish 50, Test run id:911744
Test results publishing 50, remaining: 0. Test run id: 911744
Publishing test results to test run '911743'.
TestResults To Publish 71, Test run id:911743
Test results publishing 71, remaining: 0. Test run id: 911743
Published Test Run : https://dev.azure.com/xxxxx/Runs?runId=911742&_a=runCharts
Published Test Run : https://dev.azure.com/xxxxx/Runs?runId=911744&_a=runCharts
Published Test Run : https://dev.azure.com/xxxxx/Runs?runId=911743&_a=runCharts

Publish step has this warning:

##[warning]Multiple file or directory matches were found. Using the first match: /azp/_work/_temp/1c8c6bf9-96be-405d-b0b5-b852eb474a1b/coverage.cobertura.xml

As a result of this, Build Quality checks has very low number of Covered lines


Solution

  • You can use extension ReportGenerator to merge code coverage reports.

    - task: reportgenerator@5
      inputs:
        reports: '**/coverage.cobertura.xml'
        targetdir: 'coveragereport'
        publishCodeCoverageResults: true
    - task: BuildQualityChecks@9
      displayName: 'Check build quality'
      inputs:
        checkCoverage: true
        coverageFailOption: 'fixed'
        coverageType: 'lines'
        coverageThreshold: '60'
    

    Add reportgenerator@5 task after dotnet test. You can set publishCodeCoverageResults to true in this task, so that you don't need to add PublishCodeCoverageResults@1 task.