.netdockerspecflowvstestvsdoc

How do I copy test report from docker to Agent when my dotnet tests fail in Docker Build?


I am trying to run my Automation "Specflow+ Runner" C# Test Cases using docker. I have a master.yml file which uses the Shared Agent and it builds docker image using Dockerfile.

master.yml

jobs:
  - job: run_docker_build
    displayName: Run UI Build
    steps:
#------------------------------------------
# Build Docker
#------------------------------------------
      - task: Docker@2
        displayName: DockerBuild
        env:
         COMPOSE_DOCKER_CLI_BUILD: 1
         DOCKER_BUILDKIT: 1
        inputs:
         containerRegistry: "XXX"
         repository: run-automation-ui
         command: build
         Dockerfile: '**/Dockerfile'
         tags: |
          latest
          $(Build.BuildId)

#------------------------------------------
# Copy the Results from Build Image
#------------------------------------------
      - pwsh: |
         $id=docker images --filter "label=Test=True" -q | Select-Object -First 1
         docker create --name testcontainer $id
         docker cp testcontainer:/src/AutomationTests/TestResults ./TestResults
         docker cp testcontainer:/src/AutomationTests/TestResults/Screenshots ./Screenshots
         docker rm testcontainer
        displayName: 'Copy Test Results'
        condition: always()
#------------------------------------------
# Publishing Test Results
#------------------------------------------
      - task: PublishTestResults@2
        inputs:
         testResultsFormat: 'VSTest'
         testResultsFiles: '**/*.trx'
         searchFolder: '$(System.DefaultWorkingDirectory)/TestResults'
         publishRunAttachments: true
        displayName: 'Publish Test Results' 
        condition: always()

Docker File

LABEL Test=True
##Method one (#Even if test fails the build task will finish and in later stage I can copy the Results ) 
#RUN dotnet test -c Release --no-build --logger "trx;LogFileName=/src/AutomationTests/TestResults/test_results.trx" --filter TestCategory=DockerTest || true 

#Method Two (#Even if test fails the build task will finish and in later stage I can copy the Results )
#RUN dotnet test -c Release --no-build --logger "trx;LogFileName=/src/AutomationTests/TestResults/test_results.trx" --filter TestCategory=DockerTest; exit 0 

#Method Three (#Even if test fails the build task will finish and in later stage I can copy the Results )
#RUN dotnet test -c Release --no-build --logger "trx;LogFileName=/src/AutomationTests/TestResults/test_results.trx" --filter TestCategory=DockerTest; \
#echo $? > /dotnet.exitcode;
#RUN exit $(cat /dotnet.exitcode)

#Method Four (if test fails means still i need to able to copy the results )
#RUN dotnet test -c Release --no-build --logger "trx;LogFileName=/src/AutomationTests/TestResults/test_results.trx" --filter TestCategory=DockerTest

I am looking how can I copy my results if any Tests failed in Dockerfile means


Solution

  • Rather than running dotnet test in a RUN instruction, you can define it as an ENTRYPOINT so that whenever you execute the container, it runs the tests. That integrates the exit code behavior directly with the running of the container.

    This can be done with a multi-stage Dockerfile, dedicating a specific stage for testing purposes.

    Here's an example Dockerfile taken from that demonstrates this pattern (taken from https://github.com/dotnet/dotnet-docker/blob/e576a1e988bc9320ba971866b01404f4d040afb7/samples/complexapp/Dockerfile):

    FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
    WORKDIR /source
    
    # copy csproj and restore as distinct layers
    COPY complexapp/*.csproj complexapp/
    COPY libfoo/*.csproj libfoo/
    COPY libbar/*.csproj libbar/
    RUN dotnet restore complexapp/complexapp.csproj
    
    # copy and build app and libraries
    COPY complexapp/ complexapp/
    COPY libfoo/ libfoo/
    COPY libbar/ libbar/
    WORKDIR /source/complexapp
    RUN dotnet build -c release --no-restore
    
    # test stage -- exposes optional entrypoint
    # target entrypoint with: docker build --target test
    FROM build AS test
    WORKDIR /source/tests
    
    COPY tests/*.csproj .
    RUN dotnet restore tests.csproj
    
    COPY tests/ .
    RUN dotnet build --no-restore
    
    ENTRYPOINT ["dotnet", "test", "--logger:trx", "--no-restore", "--no-build"]
    
    FROM build AS publish
    RUN dotnet publish -c release --no-build -o /app
    
    # final stage/image
    FROM mcr.microsoft.com/dotnet/runtime:6.0
    WORKDIR /app
    COPY --from=publish /app .
    ENTRYPOINT ["dotnet", "complexapp.dll"]
    

    To use this test stage, you would first need to run docker build, targeting that specific stage:

    docker build --target test -t tester .
    

    When you run the container, you can volume mount a path from the host to the container so that you can have access to the test results. Here's an example using the paths you indicated in your question:

    docker run --rm -v $(System.DefaultWorkingDirectory)/TestResults:/src/AutomationTests/TestResults tester