multithreadingpowershelljobs

How do I assign values to variables in PowerShell ThreadJobs?


ThreadJobs has access to the same environment as it was started in.
But normally, PowerShell will respond with a syntax error when trying to change variables from the parent level.

The documentation MS Learn - about_Thread_Jobs has som insights but nothing I could find useful.

The example below illustrates the issue when trying to use plain PowerShell variables.

[Array]$Numbers = @()

foreach ($i in 0..11) {
    $Jobs = Start-ThreadJob {
        $Using:Numbers += $using:i
    }
}

$Jobs | Wait-Job | Remove-Job

$Numbers

ParserError: 
Line |
   6 |          $Using:Numbers += $using:i
     |          ~~~~~~~~~~~~~~~
     | The assignment expression is not valid. The input to an assignment operator must 
     | be an object that is able to accept assignments, such as a variable or a property.

Solution

  • As the threads are run in parallell, there has to be some kind of method that prevents the parent object from getting corrupted or a ThreadJob from failing if two or more threads tries to perform operations at the exact same time on the object.

    After wrestling several days with the concept of thread safe execution (and getting great help with the patience of Santiago Squarzon and others, my own conclusion is:

    All the operation in the thread has to be made thread safe (hence the name).

    1. Don't try setting values with Using: if the objects are "plain" unless you can guarantee the value has been locked
    2. Even if using thread safe objects, don't try to both read and write of the value unless you can guarantee the value has been locked in between the two operations
    3. Only use the provided thread safe methods for manipulating data in the thread safe objects unless you can guarantee the value has been locked

    In .Net, I found two thread safe classes you can work with

    (There is one class name available per T-reference.)

    None of the classes has a thread safe method for incrementing values.

    So a thread safe ThreadJob in PowerShell 7.x, only adding new items to parent objects, might look like this

    $SafeNumbers = [System.Collections.Concurrent.ConcurrentBag[object]]::new()
    
    foreach ($i in 0..11) {
        $Thread = Start-ThreadJob {
            ($Using:SafeNumbers).Add($Using:i)
        }
    }
    
    Get-Job | Wait-Job | Remove-Job
    
    $SafeNumbers.ToArray()
    
    11
    10
    9
    8
    7
    6
    5
    4
    3
    2
    1
    0
    
    

    The order of the output is of course not guaranteed.