powershellescapingwinscpquotingpowershell-7.3

PowerShell 7.3.0 breaking command invocation


I use WinSCP within a Powershell script. It suddenly stopped working. After a while I could figure out that the problem appeared from a more recent version of PowerShell:

Reduced code:

& winscp `
    /log `
    /command `
        'echo Connecting...' `
        "open sftp://kjhgk:jkgh@example.com/ -hostkey=`"`"ssh-ed25519 includes spaces`"`"" 

Error message using v7.2.7

Host "example.com" does not exist.

Errror message using v7.3.0

Too many parameters for command 'open'.

As you can see with v7.3.0 WinSCP receives different input depending on the version of PS. I found out that the difference has something to do with the spaces in the hostkey. If they are omitted v7.3.0 outputs the same error.

What change to PowerShell caused this, and how can I fix it? (How can I debug such issues? I played a bit around with escaping, but the strings look the same no matter the version, no obvious breaking change that could be responsible)


Solution

  • Version 7.3.0 of PowerShell (Core) introduced a breaking change with respect to how arguments with embedded " characters (and empty-string arguments)[1] are passed to external programs, such as winscp:[2]

    While this change is mostly beneficial, because it fixes behavior that was fundamentally broken since v1 (this answer discusses the old, broken behavior), it also invariably breaks existing workarounds that build on the broken behavior, except those for calls to batch files and the WSH CLIs (wscript.exe and cscript.exe) and their associated script files (with file-name extensions such as .vbs and .js).

    To make existing workarounds continue to work, set the $PSNativeCommandArgumentPassing preference variable (temporarily) to 'Legacy':

    # Note: Enclosing the call in & { ... } makes it execute in a *child scope*
    #       limiting the change to $PSNativeCommandArgumentPassing to that scope.
    & {
      $PSNativeCommandArgumentPassing = 'Legacy'
      & winscp `
        /log `
        /command `
            'echo Connecting...' `
            "open sftp://kjhgk:jkgh@example.com/ -hostkey=`"`"ssh-ed25519 includes spaces`"`"" 
    }
    

    Unfortunately, because winscp.exe only accepts
    "open sftp://kjhgk:jkgh@example.com/ -hostkey=""ssh-ed25519 includes spaces""" on its process command line (i.e., embedded " escaped as ""), and not also the most widely used form
    "open sftp://kjhgk:jkgh@example.com/ -hostkey=\"ssh-ed25519 includes spaces\"" (embedded " escaped as \"), which the fixed behavior now employs, for winscp.exe, specifically, a workaround will continue to be required.

    If you don't want to rely on having to modify $PSNativeCommandArgumentPassing for the workaround, here are workarounds that function in both v7.2- and v7.3+ :

    Both workarounds allow you to pass arguments directly as quoted, but unfortunately also require formulating the entire (pass-through) command on a single line - except if --% is combined with splatting.


    Background information:

    The v7.3+ default $PSNativeCommandArgumentPassing value on Windows, 'Windows':

    For a summary of the impact of the breaking v7.3 change, see this comment on GitHub.

    If you have / need to write cross-edition, cross-version PowerShell code: The Native module (Install-Module Native; authored by me), has an ie function (short for: Invoke Executable), which is a polyfill that provides workaround-free cross-edition (v3+), cross-platform, and cross-version behavior in the vast majority of cases - simply prepend ie to your external-program calls.
    Caveat: In the specific case at hand it will not work, because it isn't aware that winscp.exe requires ""-escaping.


    [1] See this answer for details and workarounds.

    [2] Reverting that change in a later version and making the new behavior opt-in was briefly considered, but decided against - see GitHub issue #18694