I am trying to run a powershell script that uses the -Parallel feature in my Powershell script that is being called at DevOps pipeline, but I am getting an error message that I am not finding how to solve.
This is my pipeline:
trigger:
branches:
include:
- develop
variables:
ROOT: $(Build.SourcesDirectory)
REPOROOT: $(Build.SourcesDirectory)
OUTPUTROOT: $(REPOROOT)
WindowsContainerImage: 'onebranch.azurecr.io/windows/ltsc2022/vse2022:latest' # Docker image which is used to build the project https://aka.ms/obpipelines/containers
LinuxContainerImage: 'mcr.microsoft.com/onebranch/cbl-mariner/build:2.0'
resources:
repositories:
- repository: templates
type: git
name: OneBranch.Pipelines/GovernedTemplates
ref: refs/heads/main
extends:
template: v2/OneBranch.NonOfficial.CrossPlat.yml@templates # https://aka.ms/obpipelines/templates
parameters:
stages:
- stage: Build
jobs:
- job: Generate_Synapse_Arm_Templates
pool:
# This job must run in a Linux container because the Synapse workspace deployment task fails when run in a Windows container
type: linux
variables:
ob_outputDirectory: '$(REPOROOT)'
ob_artifactBaseName: "drop_synapse"
steps:
- task: Synapse workspace deployment@2
displayName: 'Generate Synapse Template Files'
inputs:
operation: 'validate'
ArtifactsFolder: '$(Build.SourcesDirectory)/AzureSynapseSpark/wcxsynapseusdev'
continueOnError: false
TargetWorkspaceName: wcxsynapseusdev
- task: CopyFiles@2
displayName: 'Copy Synapse Template Files'
inputs:
SourceFolder: $(Build.SourcesDirectory)/ExportedArtifacts/
Contents: '**'
TargetFolder: $(ob_outputDirectory)
# Update the Ev2 artifact version with the build number
- task: PowerShell@2
displayName: 'Set Ev2 service artifacts version'
inputs:
targetType: 'inline'
script: '$(Build.BuildNumber) | Out-File "$(Build.SourcesDirectory)\src\ev2_service_artifacts\version.txt" -Encoding ascii'
- stage: Bicep_Build
displayName: 'Build Bicep Files'
dependsOn: []
jobs:
- job: Build_Bicep
pool:
name: "AVD_1ES_POOL_PROD"
type: windows
demands: sqlpackage
hostVersion: 1ESWindows2022
variables:
ob_outputDirectory: '$(Build.SourcesDirectory)/out'
ob_pipelineartifacts_enabled: false
WindowsContainerImage: 'onebranch.azurecr.io/windows/ltsc2022/vse2022:latest'
ob_artifactBaseName: "ARM_Templates"
steps:
- checkout: self
- task: PowerShell@2
displayName: 'List files'
inputs:
targetType: 'inline'
script: 'Get-ChildItem -Path $(Build.SourcesDirectory)/boundary-data-node'
- task: PowerShell@2
displayName: 'Clear .azure folder to prevent access issues'
inputs:
targetType: 'inline'
script: |
$azureFolder = "$env:USERPROFILE\.azure"
if (Test-Path $azureFolder) {
Get-ChildItem $azureFolder -Recurse -Force | Remove-Item -Force -Recurse -ErrorAction SilentlyContinue
Write-Host "Cleared contents of $azureFolder instead of deleting it."
}
else {
Write-Host "$azureFolder does not exist."
}
- task: PowerShell@2
displayName: 'Format Bicep files'
inputs:
pwsh: true
targetType: 'filepath'
filePath: '$(Build.SourcesDirectory)/boundary-data-node/.scripts/BuildBicep.ps1'
arguments: 'format -TemplatePath "$(Build.SourcesDirectory)/boundary-data-node/src/ev2_service_artifacts/templates" -ParameterPath "$(Build.SourcesDirectory)/boundary-data-node/src/parameters"'
- task: PowerShell@2
displayName: 'Lint Bicep files'
inputs:
pwsh: true
filePath: '$(Build.SourcesDirectory)/boundary-data-node/.scripts/BuildBicep.ps1'
arguments: 'lint -TemplatePath "$(Build.SourcesDirectory)/boundary-data-node/src/ev2_service_artifacts/templates" -ParameterPath "$(Build.SourcesDirectory)/boundary-data-node/src/parameters"'
- task: PowerShell@2
displayName: 'Build Bicep files'
inputs:
pwsh: true
filePath: '$(Build.SourcesDirectory)/boundary-data-node/.scripts/BuildBicep.ps1'
arguments: 'build -TemplatePath "$(Build.SourcesDirectory)/boundary-data-node/src/ev2_service_artifacts/templates" -ParameterPath "$(Build.SourcesDirectory)/boundary-data-node/src/parameters"'
This is my error:
Starting: Format Bicep files (windows_build_container)
==============================================================================
Task : PowerShell
Description : Run a PowerShell script on Linux, macOS, or Windows
Version : 2.247.1
Author : Microsoft Corporation
Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/powershell
==============================================================================
Generating script.
Formatted command: . 'C:\__w\1\s\boundary-data-node\.scripts\BuildBicep.ps1' format -TemplatePath "C:\__w\1\s/boundary-data-node/src/ev2_service_artifacts/templates" -ParameterPath "C:\__w\1\s/boundary-data-node/src/parameters"
========================== Starting Command Output ===========================
"C:\powershell-7\pwsh.exe" -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command ". 'C:\__w\_temp\4b0a0cb6-0a48-449f-ae08-22d7a1d338af.ps1'"
No .git directory found in the directory hierarchy.
Task started at: 03/31/2025 18:43:53
Running Format task...
--------------------------
Processing templates files from folder 'C:\__w\1\s\boundary-data-node\src\ev2_service_artifacts\templates':
- Formatting file: 'C:\__w\1\s\boundary-data-node\src\ev2_service_artifacts\templates\adf.deploymentTemplate.bicep'
- Formatting file: 'C:\__w\1\s\boundary-data-node\src\ev2_service_artifacts\templates\applicationInsights.deploymentTemplate.bicep'
- Formatting file: 'C:\__w\1\s\boundary-data-node\src\ev2_service_artifacts\templates\logAnalyticsWorkspace.deploymentTemplate.bicep'
- Formatting file: 'C:\__w\1\s\boundary-data-node\src\ev2_service_artifacts\templates\machineLearningWorkspace.deploymentTemplate.bicep'
- Formatting file: 'C:\__w\1\s\boundary-data-node\src\ev2_service_artifacts\templates\machineLearningWorkspaceDataStores.deploymentTemplate.bicep'
- Formatting file: 'C:\__w\1\s\boundary-data-node\src\ev2_service_artifacts\templates\roleAssignments.deploymentTemplate.bicep'
- Formatting file: 'C:\__w\1\s\boundary-data-node\src\ev2_service_artifacts\templates\storage.deploymentTemplate.bicep'
- Formatting file: 'C:\__w\1\s\boundary-data-node\src\ev2_service_artifacts\templates\roleAssignmentsRoleId.deploymentTemplate.bicep'
- Formatting file: 'C:\__w\1\s\boundary-data-node\src\ev2_service_artifacts\templates\userManagedIdentity.deploymentTemplate.bicep'
Traceback (most recent call last): File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/_session.py", line 37, in load FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\ContainerAdministrator\\.azure\\azureProfile.json' System.Management.Automation.RemoteException During handling of the above exception, another exception occurred: System.Management.Automation.RemoteException Traceback (most recent call last): File "<frozen runpy>", line 198, in _run_module_as_main File "<frozen runpy>", line 88, in _run_code File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/__main__.py", line 30, in <module> File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/__init__.py", line 929, in get_default_cli File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/__init__.py", line 80, in __init__ File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/_session.py", line 50, in load File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/_session.py", line 54, in save PermissionError: [Errno 13] Permission denied: 'C:\\Users\\ContainerAdministrator\\.azure\\azureProfile.json'
Traceback (most recent call last): File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/_session.py", line 37, in load FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\ContainerAdministrator\\.azure\\az.json' System.Management.Automation.RemoteException During handling of the above exception, another exception occurred: System.Management.Automation.RemoteException Traceback (most recent call last): File "<frozen runpy>", line 198, in _run_module_as_main File "<frozen runpy>", line 88, in _run_code File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/__main__.py", line 30, in <module> File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/__init__.py", line 929, in get_default_cli File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/__init__.py", line 81, in __init__ File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/_session.py", line 50, in load File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/_session.py", line 54, in save PermissionError: [Errno 13] Permission denied: 'C:\\Users\\ContainerAdministrator\\.azure\\az.json'
This is my Powershell script:
param(
[string]$Mode,
[string]$TemplatePath = $null,
[string]$ParameterPath = $null
)
# Global filter for all Bicep files
$filter = "*.bicep*"
function CheckPowerShellVersion() {
$minVersion = 7
$currentVersion = $PSVersionTable.PSVersion.Major
if ($currentVersion -lt $minVersion) {
throw "Error: This script uses the -Parallel feature requiring '$minVersion' or higher.
Documentation feature: https://devblogs.microsoft.com/powershell/powershell-foreach-object-parallel-feature/)."
}
}
# Function to retrieve Bicep files based on a given path
function GetBicepFiles($path) {
return Get-ChildItem -Path $path -Filter $filter -Recurse
}
# Function to build Bicep files
function BuildBicepFiles($path, $isParam = $false) {
$files = GetBicepFiles -path $path
$files | ForEach-Object -Parallel {
$buildCommand = if ($using:isParam) { @("az", "bicep", "build-params") } else { @("az", "bicep", "build") }
$outputDir = $_.DirectoryName
if (-not (Test-Path -Path $outputDir)) {
New-Item -ItemType Directory -Path $outputDir -Force | Out-Null
}
Write-Host " - Building file: '$($_.FullName)' into '$outputDir'"
& $buildCommand[0] $buildCommand[1] $buildCommand[2] --file $_.FullName --outdir $outputDir
if ($LASTEXITCODE -ne 0) {
throw "Error: Failed to build file '$($_.FullName)'"
}
} -ThrottleLimit 50
}
# Function to format Bicep files
function FormatBicepFiles($path) {
$files = GetBicepFiles -path $path
$files | ForEach-Object -Parallel {
Write-Host " - Formatting file: '$($_.FullName)'"
$output = az bicep format --file $_.FullName 2>&1
Write-Host $output
if ($LASTEXITCODE -ne 0) {
throw "Error: Failed to format file '$($_.FullName)'"
}
} -ThrottleLimit 50
}
# Function to lint Bicep files
function LintBicepFiles($path) {
$files = GetBicepFiles -path $path
$files | ForEach-Object -Parallel {
Write-Host " - Linting file: '$($_.FullName)'"
$output = & az bicep lint --file $_.FullName 2>&1
Write-Host $output
if ($LASTEXITCODE -ne 0) {
throw "Error: Failed to lint file '$($_.FullName)'"
}
} -ThrottleLimit 50
}
# Unified function to process both templates and parameters path
function ProcessAllBicepFiles($processFunction) {
function ProcessFolder($path, $isParam = $false) {
if (Test-Path -Path $path) {
# Run the process function for the current folder
& $processFunction $path $isParam
# Recurse into subfolders
Get-ChildItem -Path $path -Directory | ForEach-Object {
ProcessFolder -path $_.FullName -isParam $isParam
}
}
}
# Process templates path
Write-Host "--------------------------"
Write-Host "Processing templates files from folder '$baseTemplatePath':"
ProcessFolder -path $baseTemplatePath -isParam $false
# Process parameters path
Write-Host " "
Write-Host "Processing parameters files from folder '$baseParameterPath'"
ProcessFolder -path $baseParameterPath -isParam $true
}
# Function to run the selected task based on mode
function RunTask($mode) {
# Ensure PowerShell version is compatible before running tasks
CheckPowerShellVersion
$overallStart = Get-Date
Write-Host "Task started at: $overallStart`n"
switch ($mode.ToLower()) {
"build" {
Write-Host "Running Build task..."
ProcessAllBicepFiles -processFunction ${function:BuildBicepFiles}
}
"format" {
Write-Host "Running Format task..."
ProcessAllBicepFiles -processFunction ${function:FormatBicepFiles}
}
"lint" {
Write-Host "Running Lint task..."
ProcessAllBicepFiles -processFunction ${function:LintBicepFiles}
}
"full" {
Write-Host "Running Full process: Format, Lint and Build..."
ProcessAllBicepFiles -processFunction ${function:FormatBicepFiles}
ProcessAllBicepFiles -processFunction ${function:LintBicepFiles}
ProcessAllBicepFiles -processFunction ${function:BuildBicepFiles}
}
default {
Write-Host "Invalid mode specified. Use 'Build', 'Format', 'Lint', or 'Full'."
}
}
$overallEnd = Get-Date
$totalElapsed = $overallEnd - $overallStart
Write-Host "`nTotal execution time: $totalElapsed"
}
function Get-GitRoot {
param (
[string]$startDir = (Get-Location)
)
$currentDir = Get-Item -Path $startDir
while ($currentDir -and -not (Test-Path -Path (Join-Path -Path $currentDir.FullName -ChildPath ".git"))) {
$currentDir = $currentDir.Parent
}
if ($currentDir) {
return $currentDir.FullName
} else {
Write-Host "No .git directory found in the directory hierarchy."
# This ensures the script does not crash if .git is missing.
return $env:SystemDrive
}
}
$gitRoot = Get-GitRoot
# Normalize paths to avoid issues with relative paths
$TemplatePath = [System.IO.Path]::GetFullPath($TemplatePath)
$ParameterPath = [System.IO.Path]::GetFullPath($ParameterPath)
# Set default paths if not provided
$baseTemplatePath = if ($TemplatePath) { $TemplatePath } else { Join-Path -Path $gitRoot -ChildPath "src\ev2_service_artifacts\templates" }
$baseParameterPath = if ($ParameterPath) { $ParameterPath } else { Join-Path -Path $gitRoot -ChildPath "src\ev2_service_artifacts\parameters" }
# Run the task based on user input
if (-not [string]::IsNullOrEmpty($Mode)) {
RunTask -mode $Mode
} else {
RunTask -mode "build"
}
The problem was in installing Az and az bicep seems like, updated tasks:
- task: PowerShell@2
displayName: 'Install Az Module'
inputs:
pwsh: true
targetType: 'inline'
script: |
Install-Module -Name Az -Force -AllowClobber
az bicep install
az bicep version