Using and Randomizing Proxies

Question

I'm just wondering how you would go about setting a specific proxy for each request?!

The following block quote is the only thing the documentation says about this. Also, the documentation only provides an example for Java...

Firefox version 48 and newer - GeckoDriver
Firefox maintains its proxy configuration in a profile. You can preset the proxy in a profile and use that Firefox Profile or you can set it on profile that is created on the fly as is shown in the following example. With GeckoDriver the proxy has to be passed through the required capabilities.

Any advice would be appreciated!

Answer 1

usually if I am using proxies with selenium I prefer something a little simple to read and understand

class Properties:
    def __init__(self):
        self.options = Options()
        self.options.headless = True
        self.options.add_argument("ignore-certificate-errors")
        self.options.add_argument("--proxy-server=http://xxx.xxx.xx.54:xx28") #sets a proxy
        self.driver = webdriver.Chrome(options=self.options)
    

What I tend to do is get several different proxies from 'sites that offer free proxies' and test them out, the ones that don't throw an error I store them in a list or file and iterate over them while initializing the selenium class. Which ever one works is the one selenium runs on, if it gets blocked there are several more to pick from. I pull proxies by scraping such sites and storing them in a file on my computer so I do not have to go back to the site the next time I want a working proxy.

Answer 2

I solved this by setting up proxies on the about:config page on Firefox. Here is the code that you need in order to do it:

devices = {
    "mobile" : "Mozilla/5.0 (Android 4.4; Tablet; rv:41.0) Gecko/41.0 Firefox/41.0",
    "desktop" : "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.246"
}
scripts = 'var prefs = Components.classes["@mozilla.org/preferences-service;1"].getService(Components.interfaces.nsIPrefBranch); prefs.setIntPref("network.proxy.type", 1); prefs.setCharPref("network.proxy.socks", "' + proxy + '"); prefs.setIntPref("network.proxy.socks_port", port); prefs.setBoolPref("dom.webnotifications.enabled", false); prefs.setCharPref("general.useragent.override", "' + devices[device] + '");'

browser.execute_script(scripts)

If you dont want to override the UA then you dont need to use devices list and just remove the last js rule set in the script.