I am trying to populate input for my fzf sript from multiple parallel processes.
fzf = subprocess.Popen(
[
"fzf",
],
stdin=subprocess.PIPE,
text=True,
)
so I want to be able write to directly to this PIPE from other proccesses.
As multiprocessing does not support pickling IO objects I used file descriptor id. Like this:
python_process = multiprocessing.Process(
target=generate_input,
args=(fzf.stdin.fileno(),),
)
Inside my process I did write like this:
os.dup(write_fd) # I thought it could help to resolve error with fd.
os.write(write_fd, "Any string".encode("utf-8"))
This was leading to the OSError: [Errno 9] Bad file descriptor
. I dont understand why I am gettings this, because all file descriptors are inherited from parent process. To ensure this, I tried to call this code inside my child process.
_fd_types = (
("REG", stat.S_ISREG),
("FIFO", stat.S_ISFIFO),
("DIR", stat.S_ISDIR),
("CHR", stat.S_ISCHR),
("BLK", stat.S_ISBLK),
("LNK", stat.S_ISLNK),
("SOCK", stat.S_ISSOCK),
)
for fd in range(100):
try:
s = os.fstat(fd)
except:
continue
for fd_type, func in _fd_types:
if func(s.st_mode):
break
else:
fd_type = str(s.st_mode)
print(f"fd: {fd}, type: {fd_type}")
This snippet printed that my file descriptor was available via os.fstat function.
I googled a lot a have not found something similar. Or at least some projects that utilize python in this way. I understand that I can approach this in other ways.
For example:
child = subprocess.Popen(
[
"my_python_program --child",
],
stdout=subprocess.PIPE,
)
fzf = subprocess.Popen(
[
"fzf",
],
stdin=child.stdout,
)
MacM1, Python3.12
But I really want to learn how to it in a more "basic" way. I will be very grateful for any info related, thank you.
Still learning, but I found solution that satisfies me at least.
if not os.path.exists(fifo_path):
os.mkfifo(fifo_path)
python_process = multiprocessing.Process(
target=child_func,
)
python_process.start()
write_fd = open(fifo_path, "w")
write_fd.close()
fifo = os.open(fifo_path, os.O_RDONLY)
fzf = subprocess.Popen(
[
"fzf",
],
stdin=fifo,
text=True,
)