Shutdown for socketserver based Python 3 server hangs

I am working on a "simple" server using a threaded SocketServer in Python 3.

I am going through a lot of trouble implementing shutdown for this. The code below I found on the internet and shutdown works initially but stops working after sending a few commands from the client via telnet. Some investigation tells me it hangs in threading._shutdown... threading._wait_for_tstate_lock but so far this does not ring a bell.

My research tells me that there are ~42 different solutions, frameworks, etc. on how to do this in different python versions. So far I could not find a working approach for python3. E.g. I love telnetsrv (https://pypi.python.org/pypi/telnetsrv/0.4) for python 2.7 (it uses greenlets from gevent) but this one does not work for python 3. So if there is a more pythonic, std lib approach or something that works reliably I would love to hear about it!

My bet currently is with socketserver but I could not figure out yet how to deal with the hanging server. I removed all the log statements and most functionality so I can post this minimal server which exposes the issue:

# -*- coding: utf-8 -*-
import socketserver
import threading

SERVER = None


def shutdown_cmd(request):
    global SERVER
    request.send(bytes('server shutdown requested\n', 'utf-8'))
    request.close()
    SERVER.shutdown()
    print('after shutdown!!')
    #SERVER.server_close()


class service(socketserver.BaseRequestHandler):
    def handle(self):
        while True:
            try:
                msg = str(self.request.recv(1024).strip(), 'utf-8')
                if msg == 'shutdown':
                    shutdown_cmd(msg, self.request)
                else:
                    self.request.send(bytes("You said '{}'\n".format(msg), "utf-8"))
            except Exception as e:
                pass


class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
    pass


def run():
    global SERVER
    SERVER = ThreadedTCPServer(('', 1520), service)
    server_thread = threading.Thread(target=SERVER.serve_forever)
    server_thread.daemon = True
    server_thread.start()
    input("Press enter to shutdown")
    SERVER.shutdown()


if __name__ == '__main__':
    run()

It would be great being able to stop the server from the handler, too (see shutdown_cmd)

Solution

I tried two solutions to implement a tcp server which runs on Python 3 on both Linux and Windows (I tried Windows 7):

using socketserver (my question) - shutdown is not working
using asyncio (posted an answer for that) - does not work on Windows

Both solutions have been based upon search results on the web. In the end I had to give up on the idea of finding a proven solution because I could not find one. Consequently I implemented my own solution (based on gevent). I post it here because I hope it will be helpful for others to avoid stuggeling the way I did.

# -*- coding: utf-8 -*-
from gevent.server import StreamServer
from gevent.pool import Pool


class EchoServer(StreamServer):

    def __init__(self, listener, handle=None, spawn='default'):
        StreamServer.__init__(self, listener, handle=handle, spawn=spawn)

    def handle(self, socket, address):
        print('New connection from %s:%s' % address[:2])
        socket.sendall(b'Welcome to the echo server! Type quit to exit.\r\n')

        # using a makefile because we want to use readline()
        rfileobj = socket.makefile(mode='rb')
        while True:
            line = rfileobj.readline()
            if not line:
                print("client disconnected")
                break
            if line.strip().lower() == b'quit':
                print("client quit")
                break
            if line.strip().lower() == b'shutdown':
                print("client initiated server shutdown")
                self.stop()
                break
            socket.sendall(line)
            print("echoed %r" % line.decode().strip())
        rfileobj.close()


srv = EchoServer(('', 1520), spawn=Pool(20))
srv.serve_forever()