I have a long build script which is a mix of "real" python code and lengthy subprocess.run() calls to things like debootstrap, wget, apt-get or build.sh.
Everything is parallelized. One thread does debootstrap followed by apt-get, while another does build.sh, then both are joined and we start more threads for combining the results and so on. Each thread logs to a dedicated file.
While looking for a generic way to log the output of subprocess.run() started by one of these threads, I came upon the following answer: https://stackoverflow.com/a/31868783/876832
Is it a bug to call subprocess.run() in a python script with more than one thread?
Is it a bug to call
subprocess.run()in a python script with more than one thread?
No. I do that all the time.
I think the answer you linked is misguided. After all, you don't even know whether subprocess.Popen() and its facades like subprocess.run() use the fork syscall (especially on Windows, they certainly don't, since that call doesn't exist there).
Sure, if you were to manually os.fork(), I'd exercise caution in a threaded program (and indeed, as the docs say, since 3.12, if Python detects you're doing that, it'll warn you).
Under the hood, though, on POSIX systems, subprocess.Popen() uses os.posix_spawn when possible, and otherwise a very careful fork_exec implementation.
Oh, and as an aside, the question you linked actually uses the Python 2.7-era low-level thread module, these days aptly called _thread, the low-level threading API. That's a different low-level beast than threading.Threads.