How to make sure a file used by a steppable pipeline is closed even in case of an exception?

This is a self-answered question. I'm going to add an answer only for PowerShell 7.3 or newer. Feel free to add answers for any previous PowerShell version.

I have a custom logging function that accepts pipeline input and uses a steppable pipeline object to correctly forward its input to Out-File. This answer provides background why a steppable pipeline is required in this case. Also see the blog post Mastering the (steppable) pipeline.

When any command of the pipeline of which my logging function is part of, throws an exception (aka script-terminating error) and the exception is caught, the file opened by Out-File doesn't get closed and any subsequent attempt to add something to the file (e. g. using Add-Content) fails with the error message:

The process cannot access the file 'C:\LogFile.log' because it is being used by another process.

Here is a reproducible example:

$LOG_FILE_PATH = Join-Path $PSScriptRoot LogFile.log

Function Test-SteppablePipeline {

    [CmdletBinding()]
    param (
        [Parameter( Mandatory, ValueFromPipeline )]
        [string] $InputObject
    )

    begin {
        Write-Host '[begin]'

        $steppablePipeline = {
            Out-File -LiteralPath $LOG_FILE_PATH
        }.GetSteppablePipeline( $MyInvocation.CommandOrigin )

        $steppablePipeline.Begin( $true )
    }

    process {
        Write-Host '[process]'
        
        $steppablePipeline.Process( $InputObject )

        $InputObject  # Forward to next command in pipeline of caller
    }

    end {
        Write-Host '[end]'

        $steppablePipeline.End()
    }
}

try {
    'foo', 'bar' | 
        ForEach-Object { throw 'my error' } |
        Test-SteppablePipeline
}
catch {
    '[catch]'
}

Write-Host '[add]'
Add-Content -LiteralPath $LOG_FILE_PATH -Value 'baz'

Output:

[begin]
[process]
[catch]
[add]
Add-Content: C:\Test.ps1:48
Line |
  48 |  Add-Content -LiteralPath $LOG_FILE_PATH -Value 'baz'
     |  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | The process cannot access the file 'C:\LogFile.log' because it is being used by another process.

As you can see by the [messages], the end block of Test-SteppablePipeline doesn't get called because of the exception, which propably is the reason why the file doesn't get closed.

A notable mention: This isn't an issue of just the steppable pipeline but an issue for any PowerShell function that obtains a handle and needs to ensure that said handle is properly disposed before exiting.

This is a big problem that got solved in PowerShell 7.3 with the clean block as zett42 points out in his helpful answer. For anyone wondering how this problem is solved in PowerShell 7.2 and previous versions, the answer is, it can't be solved with just PowerShell, the solution is to create a binary cmdlet that implements the IDisposable interface, the engine will do the rest (ensuring that .Dispose() is called invariantly, including CTRL + C or a terminating error).

A minimal repro of zett42's code using a binary cmdlet. Note that for the sake of demo, this is using Add-Type and inline C# for compiling at runtime, ideally, this cmdlet would be precompiled.

Add-Type '
using System;
using System.Management.Automation;

[Cmdlet(VerbsDiagnostic.Test, "SteppablePipeline")]
public sealed class TestSteppablePipelineCommand : PSCmdlet, IDisposable
{
    private SteppablePipeline _pipe;

    [Parameter(Mandatory = true, ValueFromPipeline = true)]
    public string InputObject { get; set; }

    protected override void BeginProcessing()
    {
        WriteInformation("[begin]", null);
        _pipe = ScriptBlock
            .Create("Out-File -LiteralPath $LOG_FILE_PATH")
            .GetSteppablePipeline(MyInvocation.CommandOrigin);

        _pipe.Begin(true);
    }

    protected override void ProcessRecord()
    {
        WriteInformation("[process]", null);
        _pipe.Process(InputObject);
        WriteObject(InputObject, enumerateCollection: true);
    }

    protected override void EndProcessing()
    {
        WriteInformation("[end]", null);
        _pipe.End();
    }

    public void Dispose()
    {
        _pipe.Dispose();
    }
}' -PassThru -IgnoreWarnings -WA 0 | Import-Module -Assembly { $_.Assembly }

$LOG_FILE_PATH = Join-Path $pwd.Path LogFile.log

try {
    0..10 |
        Test-SteppablePipeline -InformationAction Continue |
        ForEach-Object {
            if ($_ -eq 5) {
                throw 'my error'
            }
        }
}
catch {
    '[catch]'
}

Write-Host '[add]'
Add-Content -LiteralPath $LOG_FILE_PATH -Value 'baz'
Get-Content $LOG_FILE_PATH
Remove-Item $LOG_FILE_PATH