pythonmultithreadingloopsparallel-processingsimultaneous

Run multiple python scripts in parallel from master script


I'd like to run multiple python scripts in parallel and start them from a master script. I did find solutions for this in previously asked questions, however, none of these worked if the scripts running in parallel contained loops. Let's for example define two scripts.

Script 1:

array_1 = []

x = 0
while True:
    array_1.append(x)
    x = x + 1

Script 2:

array_2 = []

x = 0
while True:
    array_2.append(x)
    x = x + 1

Now I want to run both processes simultaneously. Previous solutions suggested the following code for a master script:

import script_1, script_2

exec(open(script_1))
exec(open(script_2))

While this is a solution for starting scripts from within another script, however, this will not run the two scripts in parallel. What should such a master script actually look like ?

Thanks for your suggestions!

Edit

I tried the following threading approach:

def function_1():
print('function 1 started...')
    while True:
        print('1')
        sleep(1)

def function_2():
print('function 2 started...')
    while True:
        print('2')
        sleep(1)

thread_1 = Thread(target=function_1())
thread_2 = Thread(target=function_2())
thread_1.start()
thread_2.start()


thread_1.join()
thread_2.join()
print("thread finished")

It doesn't work, only the first function gets started so I get the following output:

function 1 started...
1
1
1
1
1
1

Solution

  • When you want to spawn a new thread, you need to pass the address of the function you want the thread to execute, and not to call it. What you are doing here is essentially spawning a new thread that immediately calls function_1() which of course runs forever.

    Also, you won't be able to reach this line of code:

    print("thread finished")
    

    As the threads are executing a while loop - forever, so it is redundent..

    from time import sleep
    from threading import Thread
    
    
    def function_1():
        print('function 1 started...')
        while True:
            print('1')
            sleep(1)
    
    
    def function_2():
        print('function 2 started...')
        while True:
            print('2')
            sleep(1)
    
    
    thread_1 = Thread(target=function_1)
    thread_2 = Thread(target=function_2)
    thread_1.start()
    thread_2.start()
    
    thread_1.join()
    thread_2.join()
    # print("thread finished") - redundant