Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

how to avoid google search detect selenium webdriver as unusual behaviour?

I try to use selenium webdriver to do one search by image in google so my user didn't need to manually open the browser and paste image url there. but google say

Our systems have detected unusual traffic from your computer network. This page checks to see if it's really you sending the requests, and not a robot.

And give captcha, is there a way to avoid being detected as automation by google using selenium webdriver?

here my code:

@Before
public void setUp() throws Exception {
    driver = new FirefoxDriver();
    baseUrl = "http://images.google.com/searchbyimage?image_url=";
    driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
}

@Test
public void test2() throws Exception {
    driver.get(baseUrl + "http://somesite.com/somepicture.jpg");
    driver.findElement(By.linkText("sometext"));

    System.out.println("finish");

}

@After
public void tearDown() throws Exception {
    driver.quit();
    String verificationErrorString = verificationErrors.toString();
    if (!"".equals(verificationErrorString)) {
        fail(verificationErrorString);
    }
}
like image 232
Angga Avatar asked Mar 05 '15 14:03

Angga


Video Answer


2 Answers

It appear that google detect browser profile to determine something strange has going on or not. for example if you do private browsing with your browser(i test it with firefox and chrome), your browser profile will change to anonymous, so google will find it suspicious and request you to fill captcha.

That case also happen when you run your browser from selenium webdriver.

So you need to set the selenium driver profile to your default profile by using some code like this(currently only work on firefox)

ProfilesIni allProfiles = new ProfilesIni();
WebDriver driver = new FirefoxDriver(allProfiles.getProfile("default"));
like image 78
Angga Avatar answered Oct 09 '22 00:10

Angga


I disagree with @Angga and doubt that Google knows that you're a bot because you're NOT in your default profile.

It's more likely because of this: Can a website detect when you are using selenium with chromedriver?

like image 28
etayluz Avatar answered Oct 09 '22 02:10

etayluz