Using SimpleHTTPServer for unit testing

Question

I'm writing a Python module that wraps out a certain web service API. It's all REST, so relatively straightforward to implement.

However, I found a problem when it comes to unit testing: as I don't run the services I made this module for, I don't want to hammer them, but at the same time, I need to retrieve data to run my tests. I looked at SimpleHTTPServer, which would be OK.

I solved part of the issue I had, but now, since I can't seem to terminate the thread, I get issues with "address already in use" when launching the test application more than once.

Here's some sample code

PORT = 8001

handler = SimpleHTTPServer.SimpleHTTPRequestHandler

httpd = SocketServer.TCPServer(("", PORT), handler)

httpd_thread = threading.Thread(target=httpd.serve_forever)
httpd_thread.setDaemon(True)
httpd_thread.start()

api_data = urllib.urlopen("http://localhost:8001/post/index.json")
print "Data start:"
print json.load(api_data)

Where "index.json" is a mock JSON file I made which substitutes the real thing. How can I clean things gracefully after the program terminates?

Answer 1

Try using a subclass of TCPServer with allow_reuse_address set True:

class TestServer(SocketServer.TCPServer):
    allow_reuse_address = True

...
httpd = TestServer(("", PORT), handler)

Answer 2

Old thread but the answers here didn't help me, I'm using HTTPServer, and shutting down after each unit test (by default HTTPServer has allow_reuse_address = 1 set). However I still got the address already in use problem after calling shutdown. I fixed using:

from BaseHTTPServer import HTTPServer

class MyHTTPServer(HTTPServer):
    def shutdown(self):
        self.socket.close()
        HTTPServer.shutdown(self)

Not sure why this doesn't happen by default? May be this isn't optimal?