How to stop BaseHTTPServer.serve_forever() in a BaseHTTPRequestHandler subclass?

Question

I should start by saying that "I probably wouldn't do this myself, but I have in the past". The serve_forever (from SocketServer.py) method looks like this:

def serve_forever(self):
    """Handle one request at a time until doomsday."""
    while 1:
        self.handle_request()

You could replace (in subclass) while 1 with while self.should_be_running, and modify that value from a different thread. Something like:

def stop_serving_forever(self):
    """Stop handling requests"""
    self.should_be_running = 0
    # Make a fake request to the server, to really force it to stop.
    # Otherwise it will just stop on the next request.
    # (Exercise for the reader.)
    self.make_a_fake_request_to_myself()

Edit: I dug up the actual code I used at the time:

class StoppableRPCServer(SimpleXMLRPCServer.SimpleXMLRPCServer):

    stopped = False
    allow_reuse_address = True

    def __init__(self, *args, **kw):
        SimpleXMLRPCServer.SimpleXMLRPCServer.__init__(self, *args, **kw)
        self.register_function(lambda: 'OK', 'ping')

    def serve_forever(self):
        while not self.stopped:
            self.handle_request()

    def force_stop(self):
        self.server_close()
        self.stopped = True
        self.create_dummy_request()

    def create_dummy_request(self):
        server = xmlrpclib.Server('http://%s:%s' % self.server_address)
        server.ping()

Another way to do it, based on http://docs.python.org/2/library/basehttpserver.html#more-examples, is: instead of serve_forever(), keep serving as long as a condition is met, with the server checking the condition before and after each request. For example:

import CGIHTTPServer
import BaseHTTPServer

KEEP_RUNNING = True

def keep_running():
    return KEEP_RUNNING

class Handler(CGIHTTPServer.CGIHTTPRequestHandler):
    cgi_directories = ["/cgi-bin"]

httpd = BaseHTTPServer.HTTPServer(("", 8000), Handler)

while keep_running():
    httpd.handle_request()

In my python 2.6 installation, I can call it on the underlying TCPServer - it still there inside your HTTPServer:

TCPServer.shutdown


>>> import BaseHTTPServer
>>> h=BaseHTTPServer.HTTPServer(('',5555), BaseHTTPServer.BaseHTTPRequestHandler)
>>> h.shutdown
<bound method HTTPServer.shutdown of <BaseHTTPServer.HTTPServer instance at 0x0100D800>>
>>> 

The event-loops ends on SIGTERM, Ctrl+C or when shutdown() is called.

server_close() must be called after server_forever() to close the listening socket.

import http.server

class StoppableHTTPServer(http.server.HTTPServer):
    def run(self):
        try:
            self.serve_forever()
        except KeyboardInterrupt:
            pass
        finally:
            # Clean-up server (close socket, etc.)
            self.server_close()

Simple server stoppable with user action (SIGTERM, Ctrl+C, ...):

server = StoppableHTTPServer(("127.0.0.1", 8080),
                             http.server.BaseHTTPRequestHandler)
server.run()

Server running in a thread:

import threading

server = StoppableHTTPServer(("127.0.0.1", 8080),
                             http.server.BaseHTTPRequestHandler)

# Start processing requests
thread = threading.Thread(None, server.run)
thread.start()

# ... do things ...

# Shutdown server
server.shutdown()
thread.join()

I think you can use [serverName].socket.close()

In python 2.7, calling shutdown() works but only if you are serving via serve_forever, because it uses async select and a polling loop. Running your own loop with handle_request() ironically excludes this functionality because it implies a dumb blocking call.

From SocketServer.py's BaseServer:

def serve_forever(self, poll_interval=0.5):
    """Handle one request at a time until shutdown.

    Polls for shutdown every poll_interval seconds. Ignores
    self.timeout. If you need to do periodic tasks, do them in
    another thread.
    """
    self.__is_shut_down.clear()
    try:
        while not self.__shutdown_request:
            # XXX: Consider using another file descriptor or
            # connecting to the socket to wake this up instead of
            # polling. Polling reduces our responsiveness to a
            # shutdown request and wastes cpu at all other times.
            r, w, e = select.select([self], [], [], poll_interval)
            if self in r:
                self._handle_request_noblock()
    finally:
        self.__shutdown_request = False
        self.__is_shut_down.set()

Heres part of my code for doing a blocking shutdown from another thread, using an event to wait for completion:

class MockWebServerFixture(object):
    def start_webserver(self):
        """
        start the web server on a new thread
        """
        self._webserver_died = threading.Event()
        self._webserver_thread = threading.Thread(
                target=self._run_webserver_thread)
        self._webserver_thread.start()

    def _run_webserver_thread(self):
        self.webserver.serve_forever()
        self._webserver_died.set()

    def _kill_webserver(self):
        if not self._webserver_thread:
            return

        self.webserver.shutdown()

        # wait for thread to die for a bit, then give up raising an exception.
        if not self._webserver_died.wait(5):
            raise ValueError("couldn't kill webserver")

This method I use successfully (Python 3) to stop the server from the web application itself (a web page):

import http.server
import os
import re

class PatientHTTPRequestHandler(http.server.SimpleHTTPRequestHandler):
    stop_server = False
    base_directory = "/static/"
    # A file to use as an "server stopped user information" page.
    stop_command = "/control/stop.html"
    def send_head(self):
        self.path = os.path.normpath(self.path)
        if self.path == PatientHTTPRequestHandler.stop_command and self.address_string() == "127.0.0.1":
            # I wanted that only the local machine could stop the server.
            PatientHTTPRequestHandler.stop_server = True
            # Allow the stop page to be displayed.
            return http.server.SimpleHTTPRequestHandler.send_head(self)
        if self.path.startswith(PatientHTTPRequestHandler.base_directory):
            return http.server.SimpleHTTPRequestHandler.send_head(self)
        else:
            return self.send_error(404, "Not allowed", "The path you requested is forbidden.")

if __name__ == "__main__":
    httpd = http.server.HTTPServer(("127.0.0.1", 8080), PatientHTTPRequestHandler)
    # A timeout is needed for server to check periodically for KeyboardInterrupt
    httpd.timeout = 1
    while not PatientHTTPRequestHandler.stop_server:
        httpd.handle_request()

This way, pages served via base address http://localhost:8080/static/ (example http://localhost:8080/static/styles/common.css) will be served by the default handler, an access to http://localhost:8080/control/stop.html from the server's computer will display stop.html then stop the server, any other option will be forbidden.