Requests: Check that the proxy has been used to make the HTTP Request

Question

I've been scratching my head the whole day yesterday about this and to my surprise, can't seem to find an easy way to check this.

I am using Python's Requests library to pass my proxy such as:

def make_request(url):
    with requests.Session() as s:
        s.mount("http://", HTTPAdapter(max_retries=3))
        s.mount("https://", HTTPAdapter(max_retries=3))
        page = None
        d.rotate(-1) #d contains a dict of my proxies. this allows to rotate through the proxies everytime make_request is called.

        s.proxies = d[0]
        page = s.get(url, timeout=3)

        print('proxy used: ' + str(d[0]))
    return page.content

Problem is, I can't seem to make the request fail when the proxy is not expected to work. It seems there is always a fallback on my internet ip if the proxy is not working. For example: I tried passing a random proxy ip like 101.101.101.101:8800 or removing the ip authentication that is needed on my proxies, the request is still passed, even though it should'nt.

I thought adding the timeout parameters when passing the request would do the trick, but obviously it didn't.

So

• Why does this happen?
• How can I check from which ip a request is being made?

Answer 1

From what I have seen so far, you should use the form

s.get(url, proxies = d) 

This should use the proxies in the dict d to make a connection. This form allowed me to check with working proxies and non-working proxies the status_code

print(s.status_code)

I will update once I find out whether it just circulates over the proxies in the dict to match a working one, or one is able to actually select which one to be used.

[UPDATE] Tried to work around the dict in proxies, to use different proxy if I wanted to. However, proxies must be a dict to work. So I used a dict in the form of:

d = {"https" : 'https://' + str(proxy_ips[n].strip('\n'))}

This seems to work and allow me to use an ip I want to. Although it seems quite dull, I hope someone might come and help!

The proxies used can be seen through:

requests.utils.getproxies()

or

requests.utils.get_environ_proxies(url)

I hope that helps, obviously quite old question, but still!