pythonconcurrencypython-multiprocessing

Running a python child process concurrently doesn't seem to be concurrent

I've been tasked to update an existing python program that currently runs as a single process, to run a child process concurrently. I'm new to python concurrency techniques and wrote a simple script to see how to do this using the multiprocessing library:

import datetime
import logging
import multiprocessing as mp
import time
from multiprocessing import Process


def child_operation():
    """Run a child process."""
    name = 'child'
    period = 1
    logging.info(f"{datetime.datetime.now()}: started operation {name}")
    for i in range(10):
        logging.info(f"operation {name}, {i=}: sleeping for period {period} sec")
        time.sleep(period)
    logging.info(f"{datetime.datetime.now()}: ended operation {name}")


def main():
    logging.basicConfig(format='%(asctime)s %(message)s', level=logging.DEBUG)
    mp.set_start_method('spawn')

    main_sleep_period: int = 5

    # Spawn a process
    process = Process(target=child_operation())
    process.start()

    # Now do something in main while process is running
    logging.info(f'main: sleeping for {main_sleep_period} sec')  # XX
    time.sleep(main_sleep_period)
    logging.info('main stopped sleeping')

    process.join()


if __name__ == "__main__":
    main()

So the child process runs for about 10 seconds. I expected the line marked 'XX' in the 'main' process to run shortly after the child process starts. However I always find that this line appears after the child process completes its operation:

2024-08-23 13:32:56,946 2024-08-23 13:32:56.946205: started operation child
2024-08-23 13:32:56,946 operation child, i=0: sleeping for period 1 sec
2024-08-23 13:32:57,946 operation child, i=1: sleeping for period 1 sec
2024-08-23 13:32:58,947 operation child, i=2: sleeping for period 1 sec
2024-08-23 13:32:59,948 operation child, i=3: sleeping for period 1 sec
2024-08-23 13:33:00,949 operation child, i=4: sleeping for period 1 sec
2024-08-23 13:33:01,950 operation child, i=5: sleeping for period 1 sec
2024-08-23 13:33:02,951 operation child, i=6: sleeping for period 1 sec
2024-08-23 13:33:03,952 operation child, i=7: sleeping for period 1 sec
2024-08-23 13:33:04,953 operation child, i=8: sleeping for period 1 sec
2024-08-23 13:33:05,953 operation child, i=9: sleeping for period 1 sec
2024-08-23 13:33:06,954 2024-08-23 13:33:06.954231: ended operation child
2024-08-23 13:33:06,976 main: sleeping for 5 sec
2024-08-23 13:33:11,978 main stopped sleeping

I guess I'm doing something silly. I tried to google for examples but the ones I've seen only show how multiple child processes run concurrently, but not one where we have a main process and a child process and how they interplay.

Can someone point me to my error?

Solution

  • Perhaps you need to pass child_operation as a callable, without the (). I mean:

    Let's see how that goes