How to get the underlying socket when using Python requests

Question

I have a Python script that creates many short-lived, simultaneous connections using the requests library. I specifically need to find out the source port used by each connection and I figure I need access to the underlying socket for that. Is there a way to get this through the response object?

Answer 1

For streaming connections (those opened with the stream=True parameter), you can call the .raw.fileno() method on the response object to get an open file descriptor.

You can use the socket.fromfd(...) method to create a Python socket object from the descriptor:

>>> import requests
>>> import socket
>>> r = requests.get('http://google.com/', stream=True)
>>> s = socket.fromfd(r.raw.fileno(), socket.AF_INET, socket.SOCK_STREAM)
>>> s.getpeername()
('74.125.226.49', 80)
>>> s.getsockname()
('192.168.1.60', 41323)

For non-streaming sockets, the file descriptor is cleaned up before the response object is returned. As far as I can tell there's no way to get it in this situation.