pythonbashcluster-computingsupercomputers

how to send a bash command to command line using python and execute next command without waiting for bash command to execute?

I want to execute a 3 separate bash commands on 3 separate core processors of a node of a supercomputer. I have a python script that sends a bash command via os.system to the command line, however it seems to execute the bash command to completion before going to the next line in the python script. I want to just send bash commands to the command line one after another.

 for i in range(0, len(core), 8) :
        os.system("bpsh "+str(1)+" nohup python /home/tree_df_unchuncked.py "+str(' '.join(core[i]))+" 0.01"+" y")
        os.system("bpsh "+str(1)+" nohup python /home/tree_df_unchuncked.py "+str(' '.join(core[i+1]))+" 0.01"+" y")
        os.system("bpsh "+str(1)+" nohup python /home/tree_df_unchuncked.py "+str(' '.join(core[i+2]))+" 0.01"+" y")
Solution

  • @Mark is on the right track, but just to explain what is happening: your code isn't doing anything to run the commands in parallel. Ways to do this include, but are not limited to: