pythonmultithreadingqt

Qt/PySide6: What is the best way to implement an infinite data fetching loop with a QThread?


I am implementing a small multi-threaded GUI application with PySide6 to fetch data from a (USB-connected) sensor and to visualize the data with Qt. I.e., the use can start and stop the data fetching:

Start and stop sensor data fetching

When clicking the play button a worker object is created and moved to the QThread and the QThread is started. The internet is full of different approaches how to implement such an indefinite (data fetching loop). Here the two main approaches I've came across most often so far:

class Worker(QObject):
    def __init__(self, user_stop_event: Event)
        super().__init__()
        self.user_stop_event = user_stop_event
    
     def run(self):
         while not self.user_stop_event.isSet():
             self.fetch_data()
             # Process received signals:
             Qtcore.QApplication.processEvents()
             # A sleep is important since in Python threads cannot run really in parallel (on multicore systems) and without sleep we would block the main (GUI) thread!
             QThread.sleep(50)

     def fetch_data(self):
         ...
class Worker(QObject):
    def __init__(self):
        super().__init__()

        timer = QTimer()
        timer.timeout.connect(self.fetch_data)
        timer.start(50)

     def fetch_data(self):
        ...

Both alternatives use the same mechanism to start the thread:

thread = QThread()
worker = Worker()
worker.moveToThread(thread )
thread.started.connect(worker.run)
...

What are the pros and cons of both approaches? What is the preferred implementation?

Unfortunately, Qt's official Threading Basics website does not give clear advises here. Both are working on my side, but I am not quite sure which one shall be used as our default implementation for subsequent projects.


Solution

  • Qt really has no say in this, It depends on the communication mode of whatever API you are using for communication, and whether it supports synchronous or asynchronous IO.

    synchronous (blocking) API

    If the communication is synchronous such as pyserial or QSerialPort (blocking mode) then use Approach1 without the sleep, but add a timeout on the port. Reading from synchronous IO usually drops the GIL, so you don't need the sleep. (those two APIs drop the GIL, check the docs of any other API), a similar example is given by Qt docs blocking receiver

    class Worker(QThread):
        def __init__(self, user_stop_event: Event)
            super().__init__()
            self.user_stop_event = user_stop_event
        
         def run(self):
             while not self.user_stop_event.isSet():
                 self.fetch_data()
                 # Process received signals:
                 Qtcore.QApplication.processEvents()
    
         def fetch_data(self):
             ...
             # call Serial.read() here with moderate timeout
             # maybe send some data
    

    Asynchronous (non-blocking) API

    If you are reading from an asynchronous IO like QTcpSocket or QSerialPort (async mode) then you do Approach2 because Qt's eventloop drops the GIL when it is not executing python code, and you can interrupt the process in the middle of a transmission, which wasn't possible with synchronous IO, you don't need the timer for asynchronous IO

    class Worker(QObject):
        def __init__(self):
             super().__init__()
             # construct this inside a thread slot on "started" signal
             self.socket = QTcpSocket(self)
             self.socket.readyRead.connect(self.read_data)
             self.socket.connectToHost(...)
             # this next action will block, better connect to "connected" signal
             self.socket.waitForConnected() 
             self.socket.write(...)
    
         def read_data(self):
             ...
             # data = self.socket.readAll()
             # do something with data then send another request
    

    Directly calling sleep is almost always wrong, it either delays data reading, or delays the termination signal, your thread should typically either be blocked by the synchronous IO or by QT's event loop that is waiting for asynchronous IO to finish, or any other asynchronous IO loop like asyncio. polling a socket on a timer is equally bad for the same reason.

    If you ever do need to sleep (maybe you are only polling sensor every few milliseconds) then instead call QEventLoop.processEvents passing a deadline argument with the time to "process events", this ensures signals can interrupt your "sleep" (by calling loop.exit() inside the connected slot), and be sure to check whether this interruption happened or not after you are done "processing events". the GIL will be dropped while you wait so it is like an interruptible sleep.


    Periodic Events

    Lastly for periodic events that need to happen without drift (for example sending a notification, or polling a sensor every exactly 50ms) you should be using Approach2 with the timer, because QTimer uses OS timers to avoid drifts. (It can still drift on some platforms but it drifts very slowly)

    class Worker(QObject):
        def __init__(self):
             # construct this inside the QThread "started" signal
             super().__init__()
             timer = QTimer()
             timer.timeout.connect(self.fetch_data)
             timer.start(50)
    
         def fetch_data(self):
             ...
             # poll the sensor
             # or send a notification
    

    This ensures the function is called at 50,100,150,200ms ... etc whereas a normal loop with a sleep will drift and trigger at 52,105,160,223ms... etc