I'm using the Python requests package to send http requests. I want to add a single proxy to the requests session object. eg.
session = requests.Session() session.proxies = {...} # Here I want to add a single proxy
Currently I am looping through a bunch of proxies, and at each iteration a new session is made. I only want to set a single proxy for each iteration.
The only example I see in the documentation is:
proxies = { "http": "http://10.10.1.10:3128", "https": "http://10.10.1.10:1080", } requests.get("http://example.org", proxies=proxies)
I've tried to follow this, but to no avail. Here is my code from the script:
# eg. line = 59.43.102.33:80 r = s.get('http://icanhazip.com', proxies={'http': 'http://' + line})
But I get an error:
requests.packages.urllib3.exceptions.LocationParseError: Failed to parse 59.43.102.33:80
How is it possible to set a single proxy on a session object?
To use a proxy in Python, first import the requests package. Next create a proxies dictionary that defines the HTTP and HTTPS connections. This variable should be a dictionary that maps a protocol to the proxy URL. Additionally, make a url variable set to the webpage you're scraping from.
A Session is a single instance of two individuals communicating. It belongs to a Service and maps two Participants for a Proxy application. Sessions allow you to: Specify a unique identifier relevant to your use case, such as a job ID. See whether a given Session is closed or not and the reason it has been closed.
In addition to @neowu' answer, if you would like to set a proxy for the lifetime of a session object, you can also do the following -
import requests proxies = {'http': 'http://10.11.4.254:3128'} s = requests.session() s.proxies.update(proxies) s.get("http://www.example.com") # Here the proxies will also be automatically used because we have attached those to the session object, so no need to pass separately in each call
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With