getresponse
issues many recv
calls while reading header of an HTML request. It actually issues recv
for each byte which results in many system calls. How can it be optimized?
I verified on an Ubuntu machine with strace dump.
sample code:
conn = httplib.HTTPConnection("www.python.org")
conn.request("HEAD", "/index.html")
r1 = conn.getresponse()
strace dump:
sendto(3, "HEAD /index.html HTTP/1.1\r\nHost:"..., 78, 0, NULL, 0) = 78
recvfrom(3, "H", 1, 0, NULL, NULL) = 1
recvfrom(3, "T", 1, 0, NULL, NULL) = 1
recvfrom(3, "T", 1, 0, NULL, NULL) = 1
recvfrom(3, "P", 1, 0, NULL, NULL) = 1
recvfrom(3, "/", 1, 0, NULL, NULL) = 1
...
r = conn.getresponse(buffering=True)
On Python 3.1+ there is no buffering
parameter (it is default).
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With