First, some context: I have a Visual Studio solution containing several production class libraries and nine unit test projects. All the projects are targeting .NET 5. I am running .NET 5.0.401. All unit test projects have references to coverlet.collector
. I do not use the older coverlet.msbuild
package. From my reading, using XPlat Code Coverage
is now idiomatic for .NET Core.
I have an Azure Pipelines pipeline that I use for CI builds. As a part of this pipeline, I want to run all the unit tests, generate unit test results for upload to Pipelines, and generate code coverage results for upload to Pipelines. I've already read several blog articles and documentation that would seem to make this simple; however, I've found it's anything but.
Let's start simple by running some commands on my local workstation. For the purposes of this exercise we will assume I have already successfully built the solution using this command:
dotnet build Solution.sln --configuration Debug
If I run tests like this:
dotnet test Solution.sln --configuration Debug --no-build --no-restore --collect:"XPlat Code Coverage" --results-directory artifacts/test-results
then I see code coverage results stored in subdirectories named after GUIDs. Why Microsoft decided to do this I have no idea, but that's out of my control.
(After running this command and taking the screenshot, I deleted the test-results
directory.)
Remember that I also want to generate unit test results. To do that, I add the --logger trx
parameter to the same command line. This time, lots of other folders are created that contain what appear to be duplicate code coverage results. Additionally, I get the .trx
files I'm looking for.
You'll notice there are an additional nine code coverage reports in addition to the nine .trx
files I want.
Similar duplicate files are generated on my self-hosted Pipelines build agent. In Pipelines, if I try to execute the following task:
- task: 'PublishCodeCoverageResults@1'
displayName: 'Publish code coverage results'
inputs:
codeCoverageTool: 'Cobertura'
summaryFileLocation: 'artifacts/test-results/**/coverage.cobertura.xml'
I get an error:
##[warning]Multiple file or directory matches were found. Using the first match: C:\agent\_work\38\s\artifacts\test-results\$redacted\In\redacted\coverage.cobertura.xml
##[error]No code coverage results were found to publish.
The directory it's reporting here is not one of the GUID directories generated when I omit --logger trx
but instead one of the duplicate directories.
This answer seems to indicate I'm doing everything properly. I've also posted a comment on that answer hoping for some assistance.
I have several questions:
dotnet test
correctly in order to generate code coverage results and .trx
files at the same time?dotnet test
support this scenario?ReportGenerator
extension in the Azure Pipelines marketplace actually built into dotnet
now?If you have multiple coverage files, you need merge them first before you publish. You may achieve this with following:
- task: DotNetCoreCLI@2
displayName: "dotnet test"
inputs:
command: test
projects: "*.sln"
publishTestResults: true
arguments: -c Release --no-restore --no-build /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura
- task: reportgenerator@4
displayName: "Merge code coverage reports"
inputs:
reports: "**/coverage.cobertura.xml"
targetdir: "$(Build.ArtifactStagingDirectory)/coverlet"
reporttypes: "Cobertura"
verbosity: "Verbose"
- task: PublishCodeCoverageResults@1
displayName: "Publish code coverage results"
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: "$(Build.ArtifactStagingDirectory)/coverlet/Cobertura.xml"
Note: some parameters probably need to be adjusted to your porject (like directories where you expect to have cobertura files).
I don't have in my code specifig logger for trx and I see in logs that they are produced.
If you want to pass runsettings please check this doc - Configure a test run.