Why do sockets behave like this?

Question

So I was just messing around with sockets in python. I discovered that setting the socket option SO_RECVBUF to N makes the sockets recv buffer become 2N bytes large, according to the getsockopt function. For example:

import socket
a, b = socket.socketpair()
a.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 4096)
print a.getsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF) #prints 8192
b.send('1'*5000)
print len(a.recv(5000)) #prints 5000 instead of 4096 or something else.

a.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 8192)
print a.getsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF) #prints 16384

Can someone explain this to me? I am writing an HTTP server and I want to strictly limit the size of a request to protect my preciously scarce amount of RAM.

Answer 1

Internally this is making an OS level operation which according to man 7 socket says the following:

SO_RCVBUF

Sets or gets the maximum socket receive buffer in bytes. *The kernel doubles this value (to allow space for bookkeeping overhead) when it is set using setsockopt(2), and this doubled value is returned by getsockopt(2).** The default value is set by the /proc/sys/net/core/rmem_default file, and the maximum allowed value is set by the /proc/sys/net/core/rmem_max file. The minimum (doubled) value for this option is 256.

Liberally copied from this wonderful answer to a slightly different question: https://stackoverflow.com/a/11827867/758446