pythonmultithreadinggil

How do I emulate long-running cpu-bound function in python?


I want to make some experiments with threads and multiprocessing in Python. For that purpose I need a function which emulates long-running cpu-bound process. I want to manually set process time to about 5 secs. The function I created:

def long_running_cpu_bound_function():
    print(f"Started at {datetime.now()}")
    time.sleep(5)
    print(f"Executed at {datetime.now()}")

But when I run this function with 2 processes:

with concurrent.futures.ThreadPoolExecutor(max_workers=2) as executor:
    executor.submit(long_running_cpu_bound_function)
    executor.submit(long_running_cpu_bound_function)

I get the overall time about 5 seconds instead of expected 10 seconds, which I will have for real cpu-bound function because of GIL.

How do I emulate long-running cpu-bound process correctly?


Solution

  • Do some actual computation in a loop (search for the greatest prime number? Calculate digits of Pi? Just increment a counter?) and periodically check the elapsed time. Bail out after five seconds.

    import time
    ...
    start_time = time.time()
    while (time.time() - start_time) <= 5.0:
        ...do another computation step...
    

    That's for five seconds of real time. If you want to limit the total CPU time used by the whole process, then call time.process_time() instead of calling time.time().

    If you want to limit the CPU time used by just the one thread, then I invite you to read the documentation for the time module. (Hint: search for "thread_id" and then for "clk_id".)