azure-devops

build validation and status check policy on branch for Python code

I am working on adding validations as part of a pull request in Azure DevOps. Unit test (for Python code using pytest) pipeline will run as part of the build validation on creation of the pull request and will publish the test results and the code coverage as build artifacts. I have also put status check policy as mandatory for code coverage.

Now, I want to restrict the pull request from merging if code coverage does not meet the threshold. Since this is not .NET application, I am trying to change state of iteration to "failed" using RestAPI. This seems to work fine when the rest api part update the state and is visible as status check on pull request. But as soon as the build validation pipeline completes the state is set to "notApplicable" and status check disappears from pull request.

Below is sample code I am using

stages:
# unit test (pytest) in dev branch
- template: <template for unit test>

- stage: codeCoverageStatus
displayName: Code Coverage Status
jobs:
- job: validateCoveragePercent
  displayName: Validate Coverage Percent
  steps:
  - checkout: none
  - download: current
    artifact: drop
  - bash: |
        code_cov_threshold=40

        line_rate_multiplied=32

        status_state="failed"

        json_payload="{ \"name\": \"Code Coverage\", \"context\": { \"name\": \"codecoverage\", \"genre\": \"pytest_pipeline\" }, \"state\": \"$status_state\", \"description\":  \"Overall code coverage: $line_rate_multiplied %. Threshold is $code_cov_threshold % \", \"source\": \"pytest_pipeline\" }"
        echo $json_payload 

        url='https://dev.azure.com/{org}/{projectName}/_apis/git/repositories/$(Build.Repository.ID)/pullRequests/$(System.PullRequest.PullRequestId)/iterations/$(system.pullRequest.pullRequestIteration)/statuses?api-version=7.0'
        echo "URL is" $url

        curl --location --request POST $url \
        --header 'Authorization: Bearer $(System.AccessToken)' \
        --header 'Content-Type: application/json' \
        --data-raw "$json_payload"

      else
      .....
      fi
Solution

  • When the build validation is done, it will update the status of the code coverage. Since it is not a .NET application, it will change the status to "notApplicable", and the status check will disappear from the pull request. This is the expected behavior.

    This results in that even though you have changed the iteration status to "Failed" using the Rest API when the pipeline is running, it will still be covered to "notApplicable" when it is finished.

    To workaround the issue, you can set the Authorized identity of the status check to the build service account you are using in the pipeline.

    statu check

    This way, the status "Failed" will not be covered by the build validation pipeline.

    Test result:

    result