pythonmultithreadingexception

Gracefully exiting a child thread on uncaught main thread exception


I have a setup with a worker thread that looks as follows:

from time import sleep
from threading import Event, Thread


class MyThread(Thread):
    
    def __init__(self, *args, **kwargs):
        # Following Doug Fort: "Terminating a Thread"
        # (https://www.oreilly.com/library/view/python-cookbook/0596001673/ch06s03.html)
        self._stop_request = Event()
        super().__init__(*args, **kwargs)
    
    def run(self):
        while not self._stop_request.is_set():
            print("My thread is running")
            sleep(.1)
        print("My thread is about to stop")  # Finish my thread's job
        
    def join(self, *args, **kwargs):
        self._stop_request.set()
        super().join(*args, **kwargs)
            
            
if __name__ == "__main__":
    my_thread = MyThread()
    my_thread.start()
    sleep(2)
    raise RuntimeError("Something went wrong!")

With this, I would like to achieve the following: once there occurs any uncaught exception in the main thread (like the deliberate RuntimeError on the last line), the worker thread should "finish its job" (i.e. run the line printing "My thread is about to stop") and then exit, as well.

In practice, the following happens:

Using my_thread = MyThread(daemon=True) does not seem to provide a solution, as it forcefully closes the worker thread immediately, without letting it finish its job. The only working version on Windows thus seems to be: once the worker thread has been started, wrap everything else into a try–except block, thus:

if __name__ == "__main__":
    my_thread = MyThread()
    my_thread.start()
    try:
        sleep(2)
        raise RuntimeError("Something went wrong!")
    except:
        my_thread.join()

This, however, looks somewhat clumsy. Also, I do not see why it should be necessary on Windows only. Am I missing something? Is there a better solution?

Edit: On a non-WSL Linux (Python 3.9 on Ubuntu 20.04), I experienced similar behavior as under Windows; that is, the worker thread continues after the RuntimeError – but at least I can use a keyboard interrupt here. So, it does not seem to be Windows-only behavior, but maybe hints at my expectations just being wrong (after all, no one ever explicitly calls my_thread.join() in the original setup, so why should its _stop_request ever be set?). My underlying question remains the same though: how do I get the worker thread to gracefully exit, as described above?


Solution

  • Originally proposed solution

    I seem to have found a solution that works system-independent. It still feels a bit clumsy, though:

    import sys
    from time import sleep
    from threading import Event, Thread
    
    
    # Monkey-patch ``sys.excepthook`` to set a flag for notifying child threads
    # about exceptions in the main thread
    exception_raised_in_main_thread = Event()
    
    def my_excepthook(type_, value, traceback):
        exception_raised_in_main_thread.set()
        sys.__excepthook__(type_, value, traceback)
    
    sys.excepthook = my_excepthook
    
    
    class MyThread(Thread):
        
        def run(self):
            while not exception_raised_in_main_thread.is_set():
                print("My thread is running")
                sleep(.1)
            print("My thread is about to stop")
            
                
    if __name__ == "__main__":
        my_thread = MyThread()
        my_thread.start()
        sleep(2)
        raise RuntimeError("Something went wrong!")
    

    By patching sys.excepthook (which, according to the documentation, seems to be no misuse – quote "The handling of such top-level exceptions can be customized by assigning another three-argument function to sys.excepthook"), I will now set an event, exception_raised_in_main_thread, to notify all child threads about any uncaught exception that happens in the main thread.

    Middle-ground solution

    There is also somewhat of a "middle-ground solution" between what I proposed in the question and what I originally proposed as an answer: (1) Provide the global exception_raised_in_main_thread event, (2) surround all __main__ code with a try-except block and set the event in case of an exception; thus:

    from time import sleep
    from threading import Event, Thread
    
    
    exception_raised_in_main_thread = Event()
    
    
    class MyThread(Thread):
        
        def run(self):
            while not exception_raised_in_main_thread.is_set():
                print("My thread is running")
                sleep(.1)
            print("My thread is about to stop")
            
                
    if __name__ == "__main__":
        try:
            my_thread = MyThread()
            my_thread.start()
            sleep(2)
            raise RuntimeError("Something went wrong!")
        except:
            exception_raised_in_main_thread.set()
    

    Notes

    In the code snippets above, I removed the other event (self._stop_request) for brevity, but it might still come in handy for terminating the worker thread under normal circumstances.