Shut down SocketServer on SIG*

Question

I'm confused as to how to properly shut down a very simple server that I'm using.

I was thinking that this should be enough:

#!/usr/bin/python

import signal
import myhandler
import SocketServer

def terminate(signal, frame):
    print "terminating on %s at %s"
    server.shutdown()

if __name__ == "__main__":

    signal.signal(signal.SIGTERM, terminate)

    server = SocketServer.TCPServer(("localhost", 9999), myhandler.MyHandler)

    server.serve_forever()

The server works OK, but when I throw SIGTERM at it, it only prints terminating on 15 at ... but does not really shut down (i.e. close all sockets and exit).

Now pydoc explains it

 shutdown(self)
     Stops the serve_forever loop.

     Blocks until the loop has finished. This must be called while
     serve_forever() is running in another thread, or it will
     deadlock.

but this is where I'm getting lost, since I'm hardly even getting to wrap my head around threaded programming. For now I need just a simple TCP echo server that I'm able to killall and start any time (which fails now due to leftover LISTENING sockets).

So what is the correct way to achieve this?

Answer 1

Disclaimer: I have 0, nil, null, none, no experience with python. Disclaimer 2: I, in no way, think that your server is "the way to go" when it comes to...anything server related, not even for the most basic things or anything outside school homework stuff; it might be a decent sample to help people learn the basics but it is, at the same time, misleading and wrong on so many levels I lost count.

Back to your problem. I took your code and modified it to work as intended:

#!/usr/bin/python

import signal
import SocketServer
import threading
import thread

class DummyServer(SocketServer.BaseRequestHandler):
    def handle(self):
        data = self.request.recv(1024)
        self.request.send(data)
        return

def shutdownHandler(msg,evt):
    print "shutdown handler called. shutting down on thread id:%x"%(id(threading.currentThread()))
    server.shutdown()
    print "shutdown complete"
    evt.set()
    return

def terminate(signal,frame):
    print "terminate handle on thread id:%x"%(id(threading.currentThread()))
    t = threading.Thread(target = shutdownHandler,args = ('SIGTERM received',doneEvent))
    t.start()

if __name__ == "__main__":
    doneEvent = threading.Event()
    print "main thread id:%x"%(id(threading.currentThread()))
    signal.signal(signal.SIGTERM, terminate)
    server = SocketServer.TCPServer(("localhost",9999), DummyServer)
    server.serve_forever()
    doneEvent.wait()

You should check the code for SocketServer, especially the server_forever() and shutdown() methods. You should also try to learn about threads and how to do any kind of communication/signaling between them. There are lots of good sources on these topics out there.

The basic thing to remember about threads is that, generally speaking, a thread can only do ONE thing at a time - your signal handler is one of those exceptions :) If a thread is stuck in server_forever(), you can't expect the same thread to be able to run your call to shutdown() too. Python (check the signals docs) will run your signal handlers on the main thread - the same one that runs the server_forever() loop from your code: calling shutdown() from within the signal handler will lead to a deadlock, as you noticed.

The way around it is to have a new thread created for the sole purpose to run shutdown(). The new thread's shutdown() call will signal the main thread's server_forever() that it's time to break the loop and exit. The main thread might even end before the thread running shutdown() is complete - generally speaking, when the main thread ends, any other threads will be suddenly killed too without having the chance to finish whatever they were doing.

The doneEvent even is there to make sure that the main thread will wait (doneEvent.wait()) until the shutdown thread completes it's work - print "shutdown complete" before exiting.

Answer 2

As a simple solution, you can call server_close() after serve_forever():

import socketserver

class StoppableServer(socketserver.TCPServer):
    def run(self):
        try:
            self.serve_forever()
        except KeyboardInterrupt:
            pass
        finally:
            # Clean-up server (close socket, etc.)
            self.server_close()

Server stoppable with Ctrl+C or SIGTERM:

server = StoppableServer(("127.0.0.1", 8080), socketserver.BaseRequestHandler)
server.run()

Server running in a thread:

server = StoppableServer(("127.0.0.1", 8080), socketserver.BaseRequestHandler)

thread = threading.Thread(None, server.run)
thread.start()

# ... do things ...

server.shutdown()
thread.join()