pythonsubprocesspipepiping

Python: replacing shell pipeline with an ssh connection


I have the following nice bash command :

cat SomePythonScript.py | ssh remote_machine 'cat - | python'

that works very fine and that I want to write in Python. I tried with 'subprocess' but did not get that far. Can someone help me ?

from subprocess import  PIPE , Popen

p1 = Popen(["cat ", "SomePythonScript.py"], stdout=PIPE)
p2 = Popen(["remote_machine"], stdin=p1.stdout, stdout=PIPE)
p3 = Popen(["cat -", "python"], stdin=p2.stdout, stdout=PIPE)

p1.stdout.close()  
p2.stdout.close()  

output = p3.communicate()[0]

I tried also with 2 processes/ pipes

from  subprocess import PIPE , Popen  
p1 = Popen([["cat", "SomePythonScript.py"], stdout=PIPE)
p2 = Popen(["remote_machine","cat", "- python"], stdin=p1.stdout,    stdout=PIPE)
p1.stdout.close() 
output = p2.communicate()[0]

I would be very glad to any help, suggestion , advices, explanation solution... Thk in advance


Solution

  • There's nothing wrong with your use of Popen. The only things that's wrong is that you are missing the ssh command, and that you are trying to run three processes when there are only two in the bash command you're trying to mimic--'cat - | python' is just an argument to the ssh command.

    The following should better mimic your bash command:

    from subprocess import  PIPE , Popen
    
    p1 = Popen(["cat", "SomePythonScript.py"], stdout=PIPE)
    p2 = Popen(["ssh", "remote_machine", "cat - | python"], stdin=p1.stdout, stdout=PIPE)
    
    p1.stdout.close()  
    
    output = p2.communicate()[0]