Multiple outputs to same csv file in Powershell

I'm trying to export my firewall rules that are specified in multiple group policy objects and would like to include the Name of the GPO in the exported file. So I tried to take my string variable $Policy and jam it into the csv file each time a new gpo is parsed but all I'm getting is the gpo name and not the fields from Get-NetFirewallRule. Of course if I remove the $policy | Out-File $env:temp\gpos.csv -Append -Force line then I get all of the fields from Get-NetFirewallRule - but they're all on a large csv file and I can't determine their source GPO.

foreach ($policy in $PolicyObjects) {
$GPO = Open-NetGPO -PolicyStore "contoso.com\$policy"
$policy | Out-File $env:temp\gpos.csv  -Append -Force
Get-NetFirewallRule -GPOSession $GPO  | 
Select Name,
DisplayName,
DisplayGroup,
@{Name='Protocol';Expression={($PSItem | Get-NetFirewallPortFilter).Protocol}},
@{Name='LocalPort';Expression={($PSItem | Get-NetFirewallPortFilter).LocalPort}},
@{Name='RemotePort';Expression={($PSItem | Get-NetFirewallPortFilter).RemotePort}},
@{Name='RemoteAddress';Expression={($PSItem | Get-NetFirewallAddressFilter).RemoteAddress}},
Enabled,
Profile,
Direction,
Action | Export-CSV $env:temp\gpos.csv -Append -Force
}
Start-Process notepad $env:temp\gpos.csv 

Seems your $PolicyObjects is a list of your group policy displaynames. I'd tighten up your code in one of the following manners.

$PolicyObjects | ForEach-Object {

    $GPO = Open-NetGPO -PolicyStore "contoso.com\$_"

    foreach($rule in Get-NetFirewallRule -GPOSession $GPO)
    {
        $portfilter    = $rule | Get-NetFirewallPortFilter
        $addressfilter = $rule | Get-NetFirewallAddressFilter

        [PSCustomObject]@{
            GPOName       = $_
            RuleName      = $rule.name
            DisplayName   = $rule.displayname
            DisplayGroup  = $rule.displaygroup
            Protocol      = $portfilter.Protocol
            LocalPort     = $portfilter.LocalPort
            RemotePort    = $portfilter.RemotePort
            RemoteAddress = $addressfilter.RemoteAddress
            Enabled       = $rule.enabled
            Profile       = $rule.profile
            Direction     = $rule.direction
            Action        = $rule.action
        }
    }
} | Export-CSV $env:temp\gpos.csv -Force

Start-Process notepad $env:temp\gpos.csv 

or

$csvdata = foreach($policy in $PolicyObjects)
{
    $GPO = Open-NetGPO -PolicyStore "contoso.com\$policy"

    foreach($rule in Get-NetFirewallRule -GPOSession $GPO)
    {
        $portfilter    = $rule | Get-NetFirewallPortFilter
        $addressfilter = $rule | Get-NetFirewallAddressFilter

        [PSCustomObject]@{
            GPOName       = $policy
            RuleName      = $rule.name
            DisplayName   = $rule.displayname
            DisplayGroup  = $rule.displaygroup
            Protocol      = $portfilter.Protocol
            LocalPort     = $portfilter.LocalPort
            RemotePort    = $portfilter.RemotePort
            RemoteAddress = $addressfilter.RemoteAddress
            Enabled       = $rule.enabled
            Profile       = $rule.profile
            Direction     = $rule.direction
            Action        = $rule.action
        }
    }
}

$csvdata | Export-CSV $env:temp\gpos.csv -Force

Start-Process notepad $env:temp\gpos.csv 

In the first one we change the outer loop to a Foreach-Object to take advantage of the pipeline and piping straight to Export-Csv.

In the second we capture all the output then export.

In both we limit the execution time by limiting the opening/writing to file to one time, limit the portfilter calls to one per rule instead of 3, and we use the [PSCustomObject] type accelerator to construct our final object instead of piping to Select-Object with calculated expressions. Both should achieve your desired result if I understood correctly.