Shutdown socketserver serve_forever() in one-thread Python application

Question

I know that socketserver has a method shutdown() which causes server to shut down but this only works in multiple threads application since the shutdown needs to be called from different thread than the thread where serve_forever() is running.

My application handles only one request at time so I do not use separate threads for handling requests and I am unable to call shutdown() because it causes deadlock (it's not in the docs but it's stated directly in the source code of socketserver).

I will paste here a simplified version of my code for better understanding:

import socketserver  class TCPServerV4(socketserver.TCPServer):   address_family = socket.AF_INET   allow_reuse_address = True  class TCPHandler(socketserver.BaseRequestHandler):   def handle(self):     try:        data = self.request.recv(4096)     except KeyboardInterrupt:        server.shutdown()  server = TCPServerV4((host, port), TCPHandler) server.server_forever() 

I am aware that this code is not working. I just wanted to show you the thing I would like to accomplish - to shutdown server and quit the application while waiting for incoming data when the user presses CtrlC.

Answer 1

You can start another thread locally, in your handler, and call shutdown from there.

Working demo:

#!/usr/bin/env python # -*- coding: UTF-8 -*-  import SimpleHTTPServer import SocketServer import time import thread  class MyHandler(SimpleHTTPServer.SimpleHTTPRequestHandler):     def do_POST(self):         if self.path.startswith('/kill_server'):             print "Server is going down, run it again manually!"             def kill_me_please(server):                 server.shutdown()             thread.start_new_thread(kill_me_please, (httpd,))             self.send_error(500)  class MyTCPServer(SocketServer.TCPServer):     def server_bind(self):         import socket         self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)         self.socket.bind(self.server_address)  server_address = ('', 8000) httpd = MyTCPServer(server_address, MyHandler) try:     httpd.serve_forever() except KeyboardInterrupt:     pass httpd.server_close() 

Few notes:

1. To kill server, do POST request to http://localhost:8000/kill_server.
2. I create function which calls server.shutdown() and run it from another thread to solve the problem we discuss.
3. I use advice from Binding Socket: “Address already in use” to make socket instantly avaliable for reuse (you can run server again without having [Errno 98] Address already in use error). With stock TCPServer you will have to wait for connection to timeout to run you server again.

Answer 2

The SocketServer library uses some weird ways of handling inherited attributes ( guessing due to use of old style classes). If you create a server and list it's protected attributes you see:

In [4]: server = SocketServer.TCPServer(('127.0.0.1',8000),Handler) In [5]: server._  server._BaseServer__is_shut_down server.__init__ server._BaseServer__shutdown_request server.__module__ server.__doc__ server._handle_request_nonblock 

If you just add the following in your request handler:

self.server._BaseServer__shutdown_request = True 

The server will shutdown. This does the same thing as calling server.shutdown(), just without waiting (and deadlocking the main thread) until it's shutdown.