Using Scrapy with VPN on Ubuntu

Question

So i am using Scrapy to crawl some websites and i want to increase my privacy on the internet and also avoid getting banned so i read that i could achieve that by using premium proxy lists like http://www.ninjasproxy.com/ or http://hidemyass.com/ or VPN or Tor.

From what i understood a paid VPN would be a good option like the one http://hidemyass.com/ offers, but i can't seem to find any code that actually shows Scrapy integrating with a VPN like hidemyass.

I only saw an example like https://github.com/aivarsk/scrapy-proxies that shows how to use proxy lists.

How do i make Scrapy work with a VPN? If i can't are proxy lists good enough to maintain anonymity?

Answer 1

A VPN is something working system wide, not something that proxy selected traffic. All your internet traffic, browser, torrent, chat etc etc will be routed through the VPN, so just connect to the VPN and run the script.