PowerShell stripping double quotes from command line arguments

Recently I have been having some trouble using GnuWin32 from PowerShell whenever double quotes are involved.

Upon further investigation, it appears PowerShell is stripping double quotes from command line arguments, even when properly escaped.

PS C:\Documents and Settings\Nick> echo '"hello"'
"hello"
PS C:\Documents and Settings\Nick> echo.exe '"hello"'
hello
PS C:\Documents and Settings\Nick> echo.exe '\"hello\"'
"hello"

Notice that the double quotes are there when passed to PowerShell's echo cmdlet, but when passed as an argument to echo.exe, the double quotes are stripped unless escaped with a backslash (even though PowerShell's escape character is a backtick, not a backslash).

This seems like a bug to me. If I am passing the correct escaped strings to PowerShell, then PowerShell should take care of whatever escaping may be necessary for however it invokes the command.

What is going on here?

For now, the fix is to escape command line arguments in accordance with these rules (which seem to be used (indirectly) by the CreateProcess API call which PowerShell uses to invoke .exe files):

To pass a double quote, escape with a backslash: \" -> "
To pass a one or more backslashes followed by a double quote, escape each backslash with another backslash and escape the quote: \\\\\" -> \\"
If not followed by a double quote, no escaping is necessary for backslashes: \\ -> \\

Note that further escaping of double quotes may be necessary to escape the double quotes in the Windows API escaped string to PowerShell.

Here are some examples, with echo.exe from GnuWin32:

PS C:\Documents and Settings\Nick> echo.exe "\`""
"
PS C:\Documents and Settings\Nick> echo.exe "\\\\\`""
\\"
PS C:\Documents and Settings\Nick> echo.exe "\\"
\\

I imagine that this can quickly become hell if you need to pass a complicated command line parameter. Of course, none of this documented in the CreateProcess() or PowerShell documentation.

Also note that this is not necessary to pass arguments with double quotes to .NET functions or PowerShell cmdlets. For that, you need only escape your double quotes to PowerShell.

Edit: As Martin pointed out in his excellent answer, this is documented in the CommandLineToArgv() function (which the CRT uses to parse the command line arguments) documentation.

Solution

With PowerShell 7.2.0, it is finally possible for arguments passed to native executables to behave as expected. This is currently an experimental feature and needs to be enabled manually.

Enable-ExperimentalFeature PSNativeCommandArgumentPassing

After that edit your PSProfile, for example, using notepad:

notepad.exe $PROFILE

Add $PSNativeCommandArgumentPassing = 'Standard' to the top of the file. You may instead also use $PSNativeCommandArgumentPassing = 'Windows' which uses the Legacy behaviour for some native executables. The differences are documented in this pull request.

Finally, restart PowerShell. Command arguments will no longer have quotes removed.

The new behaviour can be verified with this little C program:

#include <stdio.h>

int main(int argc, char** argv) {
    for (int i = 1; i < argc; i++) {
        puts(argv[i]);
    }
    return 0;
}

Compile it with gcc and pass in some arguments with quotes, like a JSON string.

> gcc echo-test.c
> ./a.exe '{"foo": "bar"}'

With the Legacy behaviour, the output is {foo: bar}. However, with the Standard option, the output becomes {"foo": "bar"}.