I have a long script. i have a function for logging:
function Log ([string]$Content){
$Date = Get-Date
Add-Content -Path $LogPath -Value ("$Date : $Content")
}
In some point at the script i have the need to run jobs in parallel. I have a list of computer names and i need to use psexec to each one of them. this should be done as jobs to to run in parallel
Start-Job -ScriptBlock {
Log "$line Has called"
$Program_List = gc $USERS_DB_PATH\$line.txt | select -Skip 1
if (Test-Connection $line -Quiet) {
ForEach ($program in $Program_List){
Log "$line $program"
#"line $line bot is $bot pwd is $pwd"
psexec \\"$line" -u bla.local\"$bot" -p $pwd cmd bla
}
}
else{
Log "Cannot Connect to $line"
}
}
#Remove-Item "$USERS_DB_PATH\$line.txt"
}
I understand this is something to do with Scope but how can I make this scriptblock see the function Log and all the neccesery variables? they all come up empty
tl;dr
Reference variables from the caller's scope via the $using: scope.
Recreate your Log function in the context of the background job, using
$function:Log = "$using:function:Log"
Start-Job -ScriptBlock {
# Required in Windows PowerShell only (if needed).
# Change to the same working directory as the caller.
Set-Location -LiteralPath ($using:PWD).ProviderPath
# Recreate the Log function.
$function:Log = "$using:function:Log"
# All variable values from the *caller*'s scope must be $using: prefixed.
Log "$using:line Has called"
# ...
}
Read on for an explanation.
See the bottom section for better alternatives to Start-Job: Start-ThreadJob and ForEach-Object -Parallel (PowerShell (Core) 7+ only).
A background job runs in an invisible PowerShell child process, i.e. a separate powershell.exe (Windows PowerShell) pwsh (PowerShell (Core) 7+) process.
Such a child process:
$PROFILE files.Conversely, this means that only the following commands are available by default in background jobs:
*.ps1 scripts, via the directories listed in the $env:PATH environment variable.$env:PSModulePath environment variable (which has a default module).Passing caller-state information to background jobs:
Variables:
While you cannot pass variables as such to background jobs, you can pass their values, using the $using: scope; in other words: you can get the value of but not update a variable in the caller's scope - see the conceptual about_Remote_Variables.
Alternatively, pass the value as an argument via Start-Job's -ArgumentList (-Args) parameter, which the -ScriptBlock argument must then access in the usual manner: either via the automatic $args variable or via explicitly declared parameters, using a param() block.
functions:
Analogously, you cannot pass a function as such, but only a function's body, and the simplest way to do that is via namespace variable notation; e.g. to get the body of function foo, use $function:foo; to pass it to a background job (or remote call), use "$using:function:foo".
Since namespace variable notation can also be used to assign values, assigning to $function:foo creates or updates a function named foo, so that $function:foo = $using:function:foo effectively recreates a foo function in the background session.
Note that while $function:foo returns the function body as a [scriptblock] instance, $using:function:foo, turns into a string during serialization (see GitHub issue #11698; however, fortunately you can also create function bodies from strings.
As such, enclosing $using:function:foo in "..." isn't strictly necessary for Start-Job; it is, however, required for Start-ThreadJob, because in the absence of serialization in thread-based parallelism, $using:function:foo is a [scriptblock] instance, but is associated with the caller's runspace and must therefore be rebuilt from a string in the job context (otherwise, state corruption can occur).
That Start-ThreadJob even allows such script-block references may be an oversight, and the PowerShell v7+ ForEach-Object -Parallel feature (which shares technical underpinning with Start-ThreadJob) explicitly disallows them, necessitating a workaround via a helper variable that first stringifies the script block in the caller's scope - see this answer.
classes
Working directory:
In Windows PowerShell background jobs use a fixed working directory: the users Documents folder. To ensure that the background job uses the same directory as the caller, call
Set-Location -LiteralPath ($using:PWD).ProviderPath as the first statement from inside the script block passed to -ScriptBlock.
In PowerShell (Core) 7+ background job now - fortunately - use the same working directory as the caller.
Caveat re type fidelity:
Start-ThreadJob:PowerShell (Core) 7+ comes with the ThreadJob module, which offers the Start-ThreadJob cmdlet; in Windows PowerShell you can install it on demand.
ForEach-Object cmdlet, via the -Parallel parameter, which executes a script block passed to it in a separate thread for each input object.Start-ThreadJob fully integrates with PowerShell's other job-management cmdlets, but uses threads (i.e. in-process concurrency) rather than child processes, which implies:
Also, the caller's working directory is inherited.
The need for $using: / -ArgumentList equally applies.
ForEach-Object -Parallel an improvement is being considered to allow copying the caller's state to the thread script blocks on an opt-in basis - see GitHub issue #12240.This answer provides an overview of ForEach-Object -Parallel and compares and contrasts Start-Job and Start-ThreadJob.