pythonsubprocesswgetshlex

wget missing url with shlex and subprocess

I'm struggling to understand why this fails with a wget: missing URL error:

import shlex
import subprocess

copy_command = "wget -O - 'http://example.com/somepath/somefile.txt?someparam=test' | sshpass -p pass ssh user@localhost -p 2222 \"cat - > /upload/somefile.txt\""

cmd = shlex.split(copy_command, posix=False)

with subprocess.Popen(
    cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE, shell=True
) as proc:
    output, error = proc.communicate()

What am I missing here? If I just give subprocess the copy_command string directly, then it works without issues.

Solution

  • To set up a pipeline requires the parent process to spawn all the programs involved and the connect (pipe) the stdio of one to another.

    The Python documentation for subprocess explains how to do this.

    It works with string argument andshell=True because then it just hands off the command line to a sub shell, and that shell handles all those details.