Azure Devops - Unable to close PR when multiple checks attached

I have a problem with closing pull requests in Azure DevOps on-premises when they contain multiple (more than 50) checks. For pull requests with many policies attached for changes or validator groups, when closing the pull request, I receive the following errors.

When trying to close a PR that includes many attached policies for builds, I get the error:

Operation failed: policy name must be succeed to update develop. (403)

and for code reviewers policies, I receive the error:

Operation failed: Required reviewers policy needs 1 approvals but it only has 0. (403).

If I disable some policies, I can close the PR, but then the build validation does not trigger and CI doesn't check code. Sample PRs that I had trouble with had around 80 attached checks. Disabling some of them allowed me to close the PR.

How can I verify why I cannot have a large number of build validations or code reviewers attached?

My version of Azure DevOps is 19.225.34309.2 (AzureDevOps2022.1).

Screen with sample error below:

I disable some policies in order to close PR. It is workaround. I would like to know what is a reason of that behaviour and how to repair it.

I have reproduced this problem on all instance azure devops. This problem occur on azure devops services as well. In order to reproduce this problem I have created 200 pipeline and 200 policies per folder in repo. After investigation all build execured properly under the hood but PR can show only 100 checks. If I have 200 policies only 100 check is visible in PR. If you want to reprodue this problem by yourself you can use below script.

First fill variables in order to adjust script to your environment and then execure script. It will create 200 pipeline and 200 polices for those pipelines.

$url = "https://onprem.azure.devops.local url to your azure devops instance"
$organization = "yourcollectionname"
$project = "yourprojectname"
$repositoryName = "repositoryname"
$agentPool = "AgenPoolName"
$token = "YourTokenWithFullAccess"

$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(":$token"))


for ($i = 1; $i -lt 200; $i++) {

    $yamlContent = @"
trigger:
- none

pool: $agentPool

steps:
- script: echo "ok"
  displayName: 'Run a one-line script'
"@

    $3digitsNumber = '{0:d3}' -f $i
    $yamlFilePath = "folder$3digitsNumber.yaml"
    $yamlContent | Out-File -FilePath $yamlFilePath
    Remove-Item -Force $yamlFilePath -ErrorAction SilentlyContinue

    $uri = "$url/$organization/$project/_apis/git/repositories?api-version=6.0"
    $repositories = Invoke-RestMethod -Uri $uri -Method Get -Headers @{Authorization=("Basic {0}" -f $base64AuthInfo)}
    $repositoryId = $repositories.value | Where-Object name -like $repositoryName | Select-Object -ExpandProperty id

    $pipelineName = "Pipeline Folder $3digitsNumber"
    $branch = "main"

    $fileContentBase64 = [Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($yamlContent))

    $uri = "$url/$organization/$project/_apis/git/repositories/$repositoryId/refs?filter=heads/$branch&api-version=6.0"
    $response = Invoke-RestMethod -Uri $uri -Method Get -Headers @{Authorization=("Basic {0}" -f $base64AuthInfo)}
    $lastCommitId = $response.value[0].objectId

    $change = @{
        "changeType" = "add"
        "item" = @{
            "path" = "/pipelines/$yamlFilePath"
        }
        "newContent" = @{
            "content" = $fileContentBase64
            "contentType" = "base64Encoded"
        }
    }
    $commit = @{
        "comment" = "Add new file via API"
        "changes" = @($change)
    }

    $body = @{
        "refUpdates" = @(@{
            "name" = "refs/heads/$branch"
            "oldObjectId" = $lastCommitId
        })
        "commits" = @($commit)
    } | ConvertTo-Json -Depth 10

    $uri = "$url/$organization/$project/_apis/git/repositories/$repositoryId/pushes?api-version=6.0"
    $response = Invoke-RestMethod -Uri $uri -Method Post -Body $body -ContentType "application/json" -Headers @{Authorization=("Basic {0}" -f $base64AuthInfo)}


    $body = @{
        "name" = $pipelineName
        "folder" = "pipelines"
        "configuration" = @{
            "type" = "yaml"
            "path" = "/pipelines/$yamlFilePath"
            "repository" = @{
                "id" = $repositoryId
                "type" = "azureReposGit"
                "name" = $repositoryId
                "defaultBranch" = "refs/heads/$branch"
            }
        }
    } | ConvertTo-Json -Depth 10

    $uri = "$url/$organization/$project/_apis/pipelines?api-version=6.0-preview.1"
    $response = Invoke-RestMethod -Uri $uri -Method Post -Body $body -ContentType "application/json" -Headers @{Authorization=("Basic {0}" -f $base64AuthInfo)}

    if ($null -ne $response) {
        Write-Output "$3digitsNumber Pipeline created successfully."
    } else {
        Write-Output "$3digitsNumber Failed to create pipeline."
    }
}

for ($i = 1; $i -lt 200; $i++) {

    $3digitsNumber = '{0:d3}' -f $i

    $uri = "$url/$organization/$project/_apis/build/definitions?api-version=6.0"
    $response = Invoke-RestMethod -Uri $uri -Method Get -Headers @{Authorization=("Basic {0}" -f $base64AuthInfo)}
    $buildId = $response.value | Where-Object name -like "Pipeline Folder $3digitsNumber" | Select-Object -ExpandProperty id


    $body = @{
        isBlocking = $true
        isEnabled = $true
        type = @{
            id = "0609b952-1397-4640-95ec-e00a01b2c241"
        }
        settings = @{
            buildDefinitionId = $buildId
            displayName = "Folder$3digitsNumber"
            queueOnSourceUpdateOnly = $true
            manualQueueOnly = $false
            validDuration = 1440
            filenamePatterns = @("/folder$3digitsNumber/*")
            scope = @(@{
                    repositoryId = $RepositoryId
                    refName = "refs/heads/$Branch"
                    matchKind = "Exact"
                }
            )
        }
    } | ConvertTo-Json -Depth 10

    $uri = "$url/$organization/$project/_apis/policy/configurations?api-version=6.0"
    try {
        $response = Invoke-RestMethod -Uri $uri -Method Post -Body $Body -ContentType "application/json" -Headers @{Authorization=("Basic {0}" -f $base64AuthInfo)}
        Write-Output "$3digitsNumber Policy created successfully."
    } catch {
        $result = $_.Exception.Response.GetResponseStream()
        $reader = New-Object System.IO.StreamReader($result)
        $reader.BaseStream.Position = 0
        $reader.DiscardBufferedData()
        $responseBody = $reader.ReadToEnd();
        write-error "$($_.Exception) $responseBody"
    }
}

When your pipeline was created clone your repo, create new branch and simulate 200 changes using below function.

function New-TestFolders {
    [CmdletBinding()]
    param (
        $Quantity = 50
    )
    
    for ($i = 1; $i -lt $Quantity; $i++) {
        $3digitsNumber = '{0:d3}' -f $i
        mkdir -Force folder$3digitsNumber
        $randNumber = Get-Random -Minimum 0 -Maximum 999999
        "test$3digitsNumber $randNumber" | Out-File -Encoding utf8 .\folder$3digitsNumber\file$3digitsNumber.txt
    }
}

New-TestFolders 200

Then push your changes and create pull request. After that you will see only 100 checks is visible in PR despite 200 policies is attached and 200 pipeline is started.

RESOLUTION:

In order to increase list of visible checks in PR perform the SQL command below on collection DB in your environment. In that sample it increase visible checks to 200:

EXEC prc_SetRegistryValue @partitionId = 1, @key = '#\Service\Git\Settings\PullRequestDetails\BuildPolicyMaximumCount\', @value = '200'

In order to confirm the registry items value perform sql command:

select * from tbl_registryitems where ChildItem like '%BuildPolicyMaximumCount%'