Shutdown for socketserver based Python 3 server hangs

Question

I am working on a "simple" server using a threaded SocketServer in Python 3.

I am going through a lot of trouble implementing shutdown for this. The code below I found on the internet and shutdown works initially but stops working after sending a few commands from the client via telnet. Some investigation tells me it hangs in threading._shutdown... threading._wait_for_tstate_lock but so far this does not ring a bell.

My research tells me that there are ~42 different solutions, frameworks, etc. on how to do this in different python versions. So far I could not find a working approach for python3. E.g. I love telnetsrv (https://pypi.python.org/pypi/telnetsrv/0.4) for python 2.7 (it uses greenlets from gevent) but this one does not work for python 3. So if there is a more pythonic, std lib approach or something that works reliably I would love to hear about it!

My bet currently is with socketserver but I could not figure out yet how to deal with the hanging server. I removed all the log statements and most functionality so I can post this minimal server which exposes the issue:

# -*- coding: utf-8 -*-
import socketserver
import threading

SERVER = None


def shutdown_cmd(request):
    global SERVER
    request.send(bytes('server shutdown requested\n', 'utf-8'))
    request.close()
    SERVER.shutdown()
    print('after shutdown!!')
    #SERVER.server_close()


class service(socketserver.BaseRequestHandler):
    def handle(self):
        while True:
            try:
                msg = str(self.request.recv(1024).strip(), 'utf-8')
                if msg == 'shutdown':
                    shutdown_cmd(msg, self.request)
                else:
                    self.request.send(bytes("You said '{}'\n".format(msg), "utf-8"))
            except Exception as e:
                pass


class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
    pass


def run():
    global SERVER
    SERVER = ThreadedTCPServer(('', 1520), service)
    server_thread = threading.Thread(target=SERVER.serve_forever)
    server_thread.daemon = True
    server_thread.start()
    input("Press enter to shutdown")
    SERVER.shutdown()


if __name__ == '__main__':
    run()

It would be great being able to stop the server from the handler, too (see shutdown_cmd)

Answer 1

shutdown() works as expected, the server has stopped accepting new connections, but python still waiting for alive threads to terminate.

By default, socketserver.ThreadingMixIn will create new threads to handle incoming connection and by default, those are non-daemon threads, so python will wait for all alive non-daemon threads to terminate.

Of course, you could make the server spawn daemon threads, then python will not waiting:

The ThreadingMixIn class defines an attribute daemon_threads, which indicates whether or not the server should wait for thread termination. You should set the flag explicitly if you would like threads to behave autonomously; the default is False, meaning that Python will not exit until all threads created by ThreadingMixIn have exited.

class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
    daemon_threads = True

But that is not the ideal solution, you should check why threads never terminate, usually, the server should stop processing connection when no new data available or client shutdown connection:

import socketserver
import threading


shutdown_evt = threading.Event()


class service(socketserver.BaseRequestHandler):
    def handle(self):
        self.request.setblocking(False)
        while True:
            try:
                msg = self.request.recv(1024)
                if msg == b'shutdown':
                    shutdown_evt.set()
                    break
                elif msg:
                    self.request.send(b'you said: ' + msg)
                if shutdown_evt.wait(0.1):
                    break
            except Exception as e:
                break


class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
    pass


def run():
    SERVER = ThreadedTCPServer(('127.0.0.1', 10000), service)
    server_thread = threading.Thread(target=SERVER.serve_forever)
    server_thread.daemon = True
    server_thread.start()
    input("Press enter to shutdown")
    shutdown_evt.set()
    SERVER.shutdown()


if __name__ == '__main__':
    run()