I want to read input from a subprocess in python, interact with it, see the results of that interaction, and then kill the process. I have the following parent and child processes.
child.py
from random import randint
x = 1
while x < 9:
x = randint(0, 10)
print("weeeee got", x)
name = input("what's your name?\n")
print("hello", name)
x = 0
while True:
x += 1
print("bad", x)
parent.py
import curio
from curio import subprocess
async def main():
p = subprocess.Popen(
["./out"],
stdout=subprocess.PIPE,
stdin=subprocess.PIPE,
)
async for line in p.stdout:
line = line.decode("ascii")
print(p.pid, "got:", line, end="")
if "what" in line and "name" in line:
out, _ = await p.communicate(input=b"yaboi\n")
print(p.pid, "got:", out.decode("ascii"), end="")
# stuff to properly kill the process ...
return
if __name__ == "__main__:
curio.run(main)
If I run this, the parent hangs on:
out, _ = await p.communicate(input=b"yaboi\n")
Removing the following section from the child fixes the issue:
x = 1
while x < 9:
x = randint(0, 10)
print("weeeee got", x)
Why doesn't it work with the infinite loop? How do I fix this?
Edit 1: putting flush=True
in the print statement does not fix the issue.
Edit 2: The processes are in the following states:
4219 13.2 1.5 493968 487936 pts/0 S+ 14:08 0:01 python parent.py
4223 100 0.0 13764 9600 pts/0 R+ 14:08 0:14 python child.py
It looks like the python code is hanging on the communicate
call.
I suspect there's something weird going on with bidirectional process communication in python's curio. I haven't tested using vanilla subprocess, but managed to get something working in bash.
#!/bin/bash
set -e
inPipe="/tmp/in-$RANDOM"
outPipe="/tmp/out-$RANDOM"
cleanup() {
set +e
rm -f "$inPipe"
rm -f "$outPipe"
}
trap cleanup EXIT SIGINT SIGTERM
mkfifo "$inPipe" "$outPipe"
a() {
X=6
while [ $X -lt 9 ]; do
X=$((RANDOM % 11))
echo yaay $X
done
local name
echo "what's your name"
read name <$inPipe
echo hello "$name"
while true; do
echo 1
done
} >>$outPipe
a&
echo bg proc started
while read -r line; do
echo got "$line"
if (exec 1>/dev/null; rg what <<< "$line" && rg name <<< "$line"); then
echo finally got "$line"
echo "don juan" >> $inPipe
echo waiting on response
read _ name
echo "got name \"$name\" back"
echo "done"
exit 0
fi
done <$outPipe
The above communicates with a background process using two named pipes. This was surprisingly finicky to write, and deadlock occurred if care wasn't taken with shell redirects. based on this experience I strongly recommend individuals attempting to do IPC in scripts to use shell as opposed to python (as it's faster to prototype and debug).
The bash hacker's wiki is a great resource for people wanting to learn bash.