Menu
I'm trying to scrape some pages that are on a website but to view the pages, I need to be connected to a VPN. My setup is as follows:
I want to run a script on pythonanywhere that connects through the VPN so that the traffic appears to be coming from Australia.
The closest answer I have found so far is:
JSoup over VPN/proxy
796
To use a proxy in Python, first import the requests package. Next create a proxies dictionary that defines the HTTP and HTTPS connections. This variable should be a dictionary that maps a protocol to the proxy URL. Additionally, make a url variable set to the webpage you're scraping from.
A VPN is similar to a proxy, but instead of working with single apps or websites, it works with every site you visit or app you access. Like a proxy, when you visit a website after first logging into a VPN, your IP address is hidden and replaced with the IP address of your VPN provider.
I see that https://www.privateinternetaccess.com/ has option to use SOCKS5 proxy. If you are using requests module for scraping you may use SOCKS5 like that:
pip install -U requests[socks] and in the script:
import requests proxies = {'http': 'socks5://user:pass@host:port', 'https': 'socks5://user:pass@host:port'} resp = requests.get('http://example.com', proxies=proxies ) If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
