pythonmultithreadingsocketstcpsocketserver

How can I open and close a thread with Python Socket Server?


I have a simple python socket server and I want my client to be able to open/ close a thread process over the duration of multiple connections (ie I don't want to keep my client-server connection open between requests). A simplified version of the server looks like this:

_bottle_thread = None # I have tried this and a class variable

class CalibrationServer(socketserver.BaseRequestHandler):

    allow_reuse_address = True
    request_params = None
    response = None

    bottle_thread = None

    def handle(self):
        self.data = self.request.recv(1024).strip()
        self.request_params = str(self.data.decode('utf-8')).split(" ")
        method = self.request_params[0]

        if method == "start": self.handle_start()
        elif method == "stop": self.handle_stop()
        else: self.response = "ERROR: Unknown request"

        self.request.sendall(self.response.encode('utf-8'))


    def handle_start(self):
        try:
            bottle = self.request_params[1]
            _bottle_thread = threading.Thread(target=start, args=(bottle,))
            _bottle_thread.start()
            self.response = "Ran successfully"
            print(_bottle_thread.is_alive(), _bottle_thread)
        except Exception as e:
            self.response = f"ERROR: Failed to unwrap: {e}"

    def handle_stop(self):
        print(_bottle_thread)
        if _bottle_thread and _bottle_thread.is_alive():
            _bottle_thread.join()  # Wait for the thread to finish
            self.response = "Thread stopped successfully"
        else:
            self.response = "No active thread to stop"



if __name__ == "__main__":
    HOST, PORT = LOCAL_IP, CAL_PORT

    with socketserver.TCPServer((HOST, PORT), CalibrationServer) as server:
        server.serve_forever()

The process starts fine, but when my connection with the server is closed, I can no longer access the _bottle_thread. It is just re-initialized as None. I only have one master computer communicating with the server and will only need one instance of start running. I have tried using class variables to hold it and using global variables. I have also tried using Threading and Forking TCP servers. How can I access that thread to shut it down? Will I need to change it so the connection is always open? Is there another way to tackle this problem? I want to use a server so I can more easily control this process because it will be running on 8 different computers at a time but am totally open to other ideas (I tried Ansible it didn't work for me). Thanks!

EDIT:

Here is my client code:

HOST, PORT = LOCAL_IP, CAL_PORT
data = " ".join(sys.argv[1:])

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
    sock.connect((HOST, PORT))
    sock.sendall(bytes(data + "\n", "utf-8"))
    received = str(sock.recv(1024), "utf-8")

print("Sent:     {}".format(data))
print("Received: {}".format(received))

It opens, sends and closes its connection with the server with a few simple arguments.

Here is my console output from the client:

(venv) path$ python cal_client.py start argument
Sent:     start argument
Received: Ran successfully
(venv) path$ python cal_client.py stop
Sent:     stop
Received: No active thread to stop

and my server:

Received from 127.0.0.1:
['start', 'argument']
Initializing
True <Thread(Thread-1, started 8236592650235098)>
Received from 127.0.0.1:
['stop']
None # the thread is showing None

Solution

  • Ah. I missed at first what was going on...

    The issue is that when you assign to _bottle_thread, you're not assigning to the global version of it (or even to the class version of it).

    It will be instructive to run this short example:

    _bottle_thread = "Global"
    
    class C:
        _bottle_thread = "Class"
    
        def meth(self):
            _bottle_thread = "Method"
            print(f"Method {_bottle_thread}")    
    
    c = C()
    c.meth()
    
    print(f"Global: {_bottle_thread}")
    print(f"Class: {c._bottle_thread}")
    

    The output is:

    Method Method
    Global: Global
    Class: Class
    

    What's going on is that you end up with three different versions of _bottle_thread due to python scoping rules: the global one you got by assigning to it outside of all other scopes, the class one you obtained by assigning to it in the class definition, and the local one you got by assigning to it inside the method. (Note that it's not possible to print the local _bottle_thread assigned inside meth -- from outside the method, that is -- as its reference count goes to 0 as soon as the method ends and it will then be destroyed.)

    You would need to use the global keyword (https://docs.python.org/3/reference/simple_stmts.html#the-global-statement) at each scope you want the global to be visible:

    def meth(self):
        global _bottle_thread
        _bottle_thread = "Method"
    

    A better solution though is simply to keep the _bottle_thread variable as an instance variable:

    class CalibrationServer(socketserver.BaseRequestHandler):
        ...
        def __init__(self):
            # (Not strictly necessary but considered best practice to 
            # initialize instance variables in the constructor)
            self._bottle_thread = None 
            
    
        def handle_start(self):
            ...
            self._bottle_thread = threading.Thread(target=start, args=(bottle,))
            ...
        def handle_stop(self):
            if self._bottle_thread and self._bottle_thread.is_alive():
                 ...