pythondjango-rest-frameworkgunicornpython-socketio

Gunicorn multiple workers + drf + socketio have only one connection


I am using socket-client using python-socketio in django-rest-framework. And drf is running as gunicorn on the centOS server with 3 workers. When I start my service, I can see only one connection. So I can't use my chat service. How can using socket-client in drf ensure that the connection is maintained properly?

[settings.py]

from socket_utils import SocketClientClass

sio = SocketClientClass()
sio.connect_to_server()

[socket_utils.py]

import os
import time
import socketio

class SocketClientClass:
    def __init__(self):
        self.sio = socketio.Client()
        self.socket_url = os.environ.get("SOCKET_URL")

    def connect_to_server(self):
        socket_connect_cnt = 0

        while not self.sio.connected:
            socket_connect_cnt += 1
            try:
                self.sio.connect(self.socket_url)
                time.sleep(0.05)
                logger.info(f"[socket_client] connect success!")
            except Exception as ex:
                logger.error(f"[socket_client] connect exception : {str(ex)}")
            if socket_connect_cnt > 1:
                break

    def disconnect_to_server(self):
        if self.sio.connected:
            try:
                self.sio.disconnect()
                time.sleep(0.05)
                logger.info(f"[socket_client] disconnect success!")
            except Exception as ex:
                logger.error(f"[socket_client] disconnect exception : {str(ex)}")

    def message(self, event, data):
        try:
            if self.sio.connected:
                self.sio.emit(event, data)
                logger.info(f"[socket_client] send message success! : {data}")
            else:
                self.connect_to_server()
                logger.error(f"[socket_client] not connected to server!")
        except Exception as ex:
            if self.sio.connected:
                self.disconnect_to_server()
            self.connect_to_server()
            logger.error(f"[socket_client] send message exception : {str(ex)}")

My server log

1) gunicorn service

● gunicorn.service - gunicorn daemon
   Loaded: loaded (/etc/systemd/system/gunicorn.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2024-06-18 09:37:14 KST; 6min ago
 Main PID: 1269661 (python3.11)
    Tasks: 7 (limit: 23684)
   Memory: 260.6M
   CGroup: /system.slice/gunicorn.service
           ├─1269661 /home/dev/anaconda3/envs/snappy/bin/python3.11 -m gunicorn --preload --workers>
           ├─1269666 /home/dev/anaconda3/envs/snappy/bin/python3.11 -m gunicorn --preload --workers>
           ├─1269667 /home/dev/anaconda3/envs/snappy/bin/python3.11 -m gunicorn --preload --workers>
           └─1269668 /home/dev/anaconda3/envs/snappy/bin/python3.11 -m gunicorn --preload --workers>

2) socket-server

...
[2024-06-18 09:37:14,061][INFO][pid:1269257] [disconnect] sid: ddOuhjZZM1bOeSUBAAAT
[2024-06-18 09:37:14,065][INFO][pid:1269257] [disconnect] sid: IyrGaoh0s7zxJ-ICAAAV
[2024-06-18 09:37:14,927][INFO][pid:1269257] [connect] sid: Jojy3GmCCCWYAppkAAAX, remote: localhost:5002

3) socket-client

...
[2024-06-18 09:37:15,019][INFO][pid:1269661] [socket_client] connect success!

Solution

  • Main problem was in gunicorn workers. If you check the 1) gunicorn service log and 3) socket-client log above, you can see that the Main pid and socket client pid are the same (1269661).

    So I added the code below to gunicorn_conf.py.

    ...
    
    def post_worker_init(worker):
        initialize_socket_client(worker)