pythondjangomultithreadingpython-daemon

Long running background thread in Django model


I'm fairly new to Python and Django, so please let me know if there is a better way to do this. What I am trying to do is have each Device (which inherits from models.Model) kick off a long running background thread which constantly checks the health of that Device. However when I run my code, it does not seem to be executing like a daemon, as the server is sluggish and continually times out. This background thread will (in most cases) run the life of the program.

Below is a simplified version of my code:

class Device(models.Model):
    active = models.BooleanField(default=True)
    is_healthy = models.BooleanField(default=True)
    last_heartbeat = models.DateTimeField(null=True, blank=True)

    def __init__(self, *args, **kwargs):
        super(Device, self).__init__(*args, **kwargs)
        # start daemon thread that polls device's health
        thread = Thread(name='device_health_checker', target=self.health_checker())
        thread.daemon = True
        thread.start()


    def health_checker(self):
        while self.active:
            if self.last_heartbeat is not None:
                time_since_last_heartbeat = timezone.now() - self.last_heartbeat
                self.is_healthy = False if time_since_last_heartbeat.total_seconds() >= 60 else True
                self.save()
                time.sleep(10)

This seems like a very simple use of threading, but every time I search for solutions, the suggested approach is to use celery which seems like overkill to me. Is there a way to get this to work without the need for something like celery?


Solution

  • As @knbk mentioned in a comment, "Every time you query for devices, a new thread will be created for each device that is returned". This is something I originally overlooked.

    However I was able to solve my issue using a single background thread that is kicked off as a Django application. This is a much simpler approach then adding a 3rd party library (like Celery).

    class DeviceApp(AppConfig):
        name = 'device_app'
    
        def ready(self):
            # start daemon thread that polls device's health
            thread = Thread(name='device_health_checker', target=self.device_health_check)
            thread.daemon = True
            thread.start()
    
    def device_health_check(self):
        while (true):
            for device in Device.objects.get_queryset():
                if device.last_heartbeat is not None:
                    time_since_last_heartbeat = timezone.now() - device.last_heartbeat
                    device.is_healthy = False if time_since_last_heartbeat.total_seconds() >= 60 else True
                    device.save()
            time.sleep(10)