Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Selenium Python Changing IP

I am writing a web scraper using Selenium for Python. The scraper is visiting the same sites many times per hour, therefore I was hoping to find a way to alter my IP every few searches. What is the best strategy for this (I am using firefox)? Is there any prewritten code/a csv of IP addresses I can switch through? I am completely new to masking IP, proxies, etc. so please go easy on me!

like image 378
Jay Ocean Avatar asked Dec 04 '15 21:12

Jay Ocean


People also ask

Can I use python with selenium?

Selenium is an open source automation testing tool that supports a number of scripting languages like Python, C#, Java, Perl, Ruby, JavaScript, etc. depending on the application to be tested, one can choose the script accordingly.


1 Answers

Try using a proxy. There are free options (not so reliable) or paid services.

from selenium import webdriver
    
def change_proxy(proxy,port):
    profile = webdriver.FirefoxProfile()
    profile.set_preference("network.proxy.type", 1)
    profile.set_preference("network.proxy.http", proxy)
    profile.set_preference("network.proxy.http_port", port)
    profile.set_preference("network.proxy.ssl", proxy)
    profile.set_preference("network.proxy.ssl_port", port)
    driver = webdriver.Firefox(profile)
    return driver
like image 52
I.shraga Avatar answered Sep 30 '22 00:09

I.shraga