Powershell script to kill it after it runs AND capture standard output

I have an executable that does a bunch of processing and then waits for a keypress. I'm trying to run this Powershell script from the Windows scheduler. Here's my original attempt:

$logFile = "E:\BatchFiles\Log\JustWaitKey.txt"
"We're about to run out test program" >> $logFile
$process = Start-Process -FilePath "E:\BatchFiles\JustWaitKey.exe" >> $logFile -WorkingDirectory E:\BatchFiles -PassThru
Start-Sleep -Seconds 5
Stop-Process -Id $process.Id
"Our test program has completed" >> $logFile

The problem here is that it throws an error on the Stop-Process step because the parameter is null.
HOWEVER, if I remove the ">> $logFile" parameter after the ...JustWaitKey.exe, the program runs and when it finishes, it is stopped!  I do see the two messages going to the $logFile.
The problem with this is that I don't get the console output from the JustWaitKey.exe program?

The redirection on Start-Process is wrong (>> $logFile), the cmdlet already has a -RedirectStandardOutput parameter for this. So what you could do is first start the process at let it redirect the output, then after killing it, you can get the content and append the additional information you wanted:

$logFile = 'E:\BatchFiles\Log\JustWaitKey.txt'
$startProcessSplat = @{
    FilePath               = 'E:\BatchFiles\JustWaitKey.exe'
    RedirectStandardOutput = $logFile
    WorkingDirectory       = 'E:\BatchFiles'
    PassThru               = $true
}
$process = Start-Process @startProcessSplat
Start-Sleep -Seconds 5
$process | Stop-Process

@(
    "We're about to run out test program"
    Get-Content $logFile -Raw
    "Our test program has completed"
) | Set-Content $logFile

If you want your process to stream its output directly to the file, appending to it, then you need to take a much more manual approach calling the .NET APIs directly:

$logFile = 'E:\BatchFiles\Log\JustWaitKey.txt'
$process = [System.Diagnostics.Process]@{
    StartInfo = [System.Diagnostics.ProcessStartInfo]@{
        FileName               = 'E:\BatchFiles\JustWaitKey.exe'
        WorkingDirectory       = 'E:\BatchFiles'
        UseShellExecute        = $false
        RedirectStandardOutput = $true
        RedirectStandardError  = $true
    }
}

$handler = { Add-Content -LiteralPath $Event.MessageData -Value $EventArgs.Data }
$stdout = Register-ObjectEvent -InputObject $process -EventName OutputDataReceived -Action $handler -MessageData $logFile
$stderr = Register-ObjectEvent -InputObject $process -EventName ErrorDataReceived -Action $handler -MessageData $logFile
Register-ObjectEvent -InputObject $process -EventName Exited -SourceIdentifier Process.Exited

try {
    Add-Content -LiteralPath $logFile -Value "We're about to run out test program"
    $null = $process.Start()
    $process.BeginOutputReadLine()
    $process.BeginErrorReadLine()
    Start-Sleep -Seconds 5
    $process.Kill()
    Wait-Event -SourceIdentifier Process.Exited | Unregister-Event
    Add-Content -LiteralPath $logFile -Value 'Our test program has completed'
}
finally {
    $process.Dispose()
    $stdout, $stderr | Unregister-Event -SourceIdentifier { $_.Name }
}