azureazure-devopsazure-pipelinessonarqubesonarqube-scan

SonarQubePublish@6 fails when running from another stage than SonarQubePrepare, saying: "Variables are missing"

I am working on a Dev Azure pipeline which is runnig couple of stages which have tasks for preparing sonar report. The important parts of stages which I want to focus on look like:

build_and_sonar_prepare:
    - task: SonarQubePrepare@6
      displayName: prepare_sonar
      inputs:
        SonarQube: ${{ parameters.SonarQube}}
        scannerMode: 'Other'
        
    - task: Bash@3
      displayName: scan_sonar
      inputs:
        targetType: 'inline'
        script: |
          mvn sonar:sonar other commands
        
sonar_publish:
     - task: SonarQubePublish@6
       inputs:
       pollingTimeoutSec: '300'

The build_and_sonar_prepare is not managed by me, its part of a template which I am reusing. It runs fine and generates proper sonar report with a link to sonar server. The stage which I have controll is sonar_publish, where I would like to only publish the results in order to see the results of the report in Azure Devops Pipeline summary (I have created separate question for that: (Azure Devops Pipeline, show link do Sonar report on the summary page of Pipeline Run)

What happens here is

  1. SonarQubePrepare is being called with a proper working connection to SonarQube ${{ parameters.SonarQube}}, which is managed by Azure and its working fine.
  2. the mvn sonar:sonar is being called, instead of SonarCubeAnalyze and unfortunately I cannot change it, because its part of the template
  3. SonarQubePublish@6 is being called, but from another stage

After I run this pipeline I am getting an error: ##[error]Variables are missing. Please make sure that you are running the Prepare and Analyze tasks before running the Publish task.

I have tried to put the SonarCubeAnalyze in my stage, before I call Publish but it also failed. I have also ensured that all files generated by sonar report are copied in the workspace of sonar_publish (using CopyFiles@2 and PublishBuildArtifacts@1 in the build_and_sonar_prepare stage and CopyFiles@2 in the sonar_publish stage) So far without success. I wonder if it is possible to Publish sonar report with this setup? I also would like to know which variables are missing and how to check if they exist in the sonar_publish stage and what are their values?

Solution

  • You must put SonarQubePrepare@6 task and SonarQubePublish@6 task in the same job.

    I can reproduce your issue when putting SonarQubePrepare@6 task and SonarQubePublish@6 task in the different jobs.

    enter image description here

    SonarQubePrepare@6 task is used to configure all the required settings before executing the build. This task is mandatory and the scope is the current job. When running the "sonar_publish" stage, it will run on a brand-new agent which hasn't been configured with SonarQube. This agent doesn't contain any info you have configured in "build_and_sonar_prepare" stage. As shown in the screenshot, the values ​​of SONARQUBE_SCANNER_PARAMS and SONAR_ENDPOINT are both "undefined".

    If you check the debug log of a working run (In the same job), you can see that the SonarQubePublish@6 task will automatically get information from the current environment.

    enter image description here

    If you can't add SonarQubePublish@6 task to "build_and_sonar_prepare" stage, you can add SonarQubePrepare@6 task to your "sonar_publish" stage and use the same service connection.

    Some docs for your reference:

    Update:

    If you can't put SonarQubePrepare@6 task and SonarQubePublish@6 task in the same job. You can parse the log to get the result URL and put it to the build summary page.

    1. Run REST API to get the log id of task SonarQubeAnalyze@6 and use this id to get the task log.
    2. Parse the log and get the result URL. Write the URL into a Markdown file.
    3. Using logging command ##vso[task.uploadsummary] to upload result URL to the build summary page via a summary Markdown file.
    PowerShell Task

    Refer to the following PowerShell scripts:

    - job: JobB
  dependsOn: JobA
  steps:
    - task: PowerShell@2
      inputs:
        targetType: 'inline'
        script: |
          $token = "$(PAT)"
          $token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($token)"))
          $head = @{ Authorization =" Basic $token" }
          $org = "{OrgName}"
          $proj = "{ProjectName}"
          $url1 ="https://dev.azure.com/$org/$proj/_apis/build/builds/$(Build.BuildId)/timeline?api-version=6.0"
          $timeline = Invoke-RestMethod -Uri $url1 -Method Get -Headers $head 
          $targetlog = $timeline.records | Where-Object {$_.name -eq "SonarQubeAnalyze"} | Select-Object 
          $logId = $targetlog.log.id
          $logId
          
          
          $url2 = "https://dev.azure.com/$org/$proj/_apis/build/builds/$(Build.BuildId)/logs/$($logId)?api-version=6.0"
          $logContent = Invoke-RestMethod -Uri $url2 -Method Get -Headers $head 
          $logLines = $logContent -split "`n"
          $targetline = $logLines | Select-String -Pattern "ANALYSIS SUCCESSFUL, you can find the results at:"
          Write-Output $targetline
          $sonarUrl = $targetline.Line
          $sonarUrl = $sonarUrl.Split("ANALYSIS SUCCESSFUL", 2)[1]
          $sonarUrl
          $sonarUrl | Out-File -FilePath Summary.md -Encoding utf8
          Write-Host "##vso[task.uploadsummary]$(System.DefaultWorkingDirectory)/Summary.md"

    Result: enter image description here

    Bash task

        - task: Bash@3
      inputs:
        targetType: 'inline'
        script: |
          token="$(PAT)"
          token=$(echo -n ":${token}" | base64)
          head="Authorization: Basic ${token}"
          org="{OrgName}"
          proj="{ProjectName}"
          url1="https://dev.azure.com/${org}/${proj}/_apis/build/builds/$(Build.BuildId)/timeline?api-version=6.0"
          
          timeline=$(curl -s -H "${head}" "${url1}")
          targetlog=$(echo "${timeline}" | jq '.records[] | select(.name=="SonarQubeAnalyze")')
          logId=$(echo "${targetlog}" | jq '.log.id')
          
          url2="https://dev.azure.com/${org}/${proj}/_apis/build/builds/$(Build.BuildId)/logs/${logId}?api-version=6.0"
          logContent=$(curl -s -H "${head}" "${url2}")
          targetline=$(echo "${logContent}" | grep "ANALYSIS SUCCESSFUL, you can find the results at:")
          sonarUrl=$(echo "${targetline}" | cut -d' ' -f5-)
          
          echo "${sonarUrl}" > Summary.md
          echo "##vso[task.uploadsummary]$(System.DefaultWorkingDirectory)/Summary.md"

    I am testing using SonarCloud, so the URL is different.