python SimpleHTTPRequestHandler server leaves socket in TIME_WAIT state after exit

Question

I have a simple server written that extends the SimpleHTTPRequestHandler

If I start and stop it without making any requests to the server, I can start back up on the same port with no problem.

When started, a netstat looks like this:

sam@hersheezy:server$ sudo netstat -na --program | grep 8001
tcp        0      0 0.0.0.0:8001            0.0.0.0:*               LISTEN      23392/python

After a request is made, netstat looks like this (even after the request has completed):

sam@hersheezy:server$ sudo netstat -na --program | grep 8001
tcp        0      0 0.0.0.0:8001            0.0.0.0:*               LISTEN      23392/python    
tcp        0      0 127.0.0.1:8001          127.0.0.1:48659         TIME_WAIT   -

Then, I kill the server using C-c and netstat looks like this (at this point I cannot restart the server because port is already in use):

 sudo netstat -na --program | grep 8001
tcp        0      0 127.0.0.1:8001          127.0.0.1:48674         TIME_WAIT   - 

I am obviously not closing something correctly. My code that sends the reply looks like the following:

"""
reply is an object that can be json encoded that is written with a response code 200
"""
def send_provider_reply(self, replyobj):
    try:
        str_reply = json.dumps(replyobj)
        self.send_response(200)
        self.send_header('Content-type', 'application/json')
        self.end_headers()
        #do we need to send a newline??
        self.wfile.write(str_reply)
    except:
        traceback.print_exc()
        self.send_err(500, 'failed after provider creation')

Answer 1

The socket option SO_LINGER does prevent a socket to go into TIME_WAIT. But TIME_WAIT is there for a reason: it should protect you from lingering packets from older connections. Therefor the default duration of TIME_WAIT is twice the network roundtrip. So, it's normal to find some older connections in TIME_WAIT.

To give some context: on the server side, with listening sockets, there is the SO_REUSEADDR socket option. It allows the listening socket to succeed at bind before the end of TIME_WAIT. For server processes, that always should listen to the same port (think: webserver at port 80, 443), this is a must.

typical python code for a server might contain something like this:

...
listener = socket(AF_INET, SOCK_STREAM)
listener.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
listener.bind((HOST, PORT))
listener.listen(32)
...