pythoncythonpython-multithreadingcythonize

A created python thread stucks when code is compiled to cython


I have been trying to figure out why the created thread doesn't work and get stuck when compiled into Cython, as it works normally with CPython...

main.py

from count import count_launcher


if __name__ == "__main__":
    tracker = {"running": True, "count": 0}
    count_launcher(tracker)

count.py

import threading
import time


def count(tracker: dict):
    start = time.time()
    while tracker["running"]:
        tracker["count"] += 1
        # While It works if I added this print statement:
        # print("Hello, world!")
    end = time.time()
    print(f"Counted to: {tracker['count']} in {end - start} seconds")


def count_launcher(tracker: dict):
    t = threading.Thread(target=count, args=(tracker,), daemon=True)
    t.start()

    print("Started the counting process...")
    time.sleep(2)
    tracker["running"] = False
    t.join()

I have tried using different python versions (3.12, 3.11) with cython but still get the same results... also there is something strange as when I add a print statement within the while loop of the count function it works! which is kinda very strange...


Solution

  • Unlike Python, Cython doesn't release the Global Interpreter Lock (GIL) at regular intervals. Thus your second thread never gets to run because it's always waiting for this lock to become available.

    Adding the print call returns control back to Python so is an opportunity for it to release the GIL.

    Adding the following lines in the loop will also periodically release the GIL and thus share time with your other thread:

    with nogil:
        pass