powershellhashtablepowershell-7.0powershell-7.2

Why do process blocks correctly return a hashtable, instead of an array of hashtables?


I am trying to sort some REST API responses and extract data from them, and found this helpful answer on how to end up with a hashtable at the end.

$res= (Invoke-RestMethod @commonParams -uri "foo" -Method GET -FollowRelLink) | ForEach-Object { $_ } | Select-Object -Property id, filename | ForEach-Object -begin {$h=@{}} -process {$h[$_.filename] = $_.id} -end {$h}

Great, so I thought I'd shorten it a bit by getting rid of the begin/process/end blocks like so:

$res = (Invoke-RestMethod @commonParams -uri "foo" -Method GET -FollowRelLink) | ForEach-Object { $_ } | Select-Object -Property id, filename | ForEach-Object { $h=@{}; $h[$_.filename] = $_.id; $h }

All good, I should now be able to reference each item by it's key value using square brackets, but that doesn't work, only referencing by dot notation.

So I went searching and found this answer, which turned out to be correct:

$res.GetType()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     Object[]                                 System.Array

But, even changing from the generic hashtable constructor to using the .Add() method didn't help:

$res = (Invoke-RestMethod @commonParams -uri "foo" -Method GET -FollowRelLink) | ForEach-Object { $_ } | Select-Object -Property id, filename | ForEach-Object { $h=@{}; $h.add($_.filename, $_.id); $h }

$res.GetType()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     Object[]                                 System.Array

I add the process blocks back in:

$res = (Invoke-RestMethod @commonParams -uri "$foo" -Method GET -FollowRelLink) | ForEach-Object { $_ } | Select-Object -Property id, filename | ForEach-Object -begin {$h=@{}} -process {$h[$_.filename] = $_.id} -end {$h}

$res.GetType()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     Hashtable                                System.Object

And it's all working as expected.

So I'd like to know, if possible, what exactly is happening when I use the process blocks as opposed to just making 'fake' multi-line functions in the pipeline?

And if anyone cares, I actually ended up avoiding the above issue completely by using Group-Object from here.


Solution

  • ForEach-Object emulates how script blocks work in the pipeline. The reason why the first and last snippets work as expected, returning a hashtable is because -Begin and -End not because of -Process which executes by default and is a mandatory parameter. If you don't use -Begin you would simply be creating and outputting a new hashtable per input object.

    0..10 | ForEach-Object -Begin {
        "executes only once, before the first input object is processed"
    } -Process {
        "processes each input object: $_"
    } -End {
        "executes only once, after the last input object is processed"
    }
    
    0..10 | & {
        begin {
            "executes only once, before the first input object is processed"
        }
        process {
            "processes each input object: $_"
        }
        end {
            "executes only once, after the last input object is processed"
        }
    }
    

    about_Functions_Advanced_Methods explain how these blocks work.

    Also your code could be simplified to this:

    $h = @{}
    Invoke-RestMethod @commonParams -uri "foo" -Method GET -FollowRelLink |
        Write-Output |
        ForEach-Object { $h[$_.filename] = $_.id }
    

    Or using Group-Object -AsHashTable as you have already found:

    $h = Invoke-RestMethod @commonParams -uri "foo" -Method GET -FollowRelLink |
        Write-Output |
        Group-Object FileName -AsHashTable
    

    Write-Output in this case may not be needed if the output from Invoke-RestMethod is already enumerated.