Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Getting 403 when using Selenium to automate checkout process

I am trying to create a script using python and selenium to automate the checkout process at bestbuy.ca.

I get all the way to the final stage where you click to review the final order, but get the following 403 forbidden message (as seen in the network response) when I try to click through to the final step.

Is there something server side that has detected that I am using selenium and preventing me to proceed?

How can I hide the fact that it is selenium being used?

These are the options I am using for selenium:

options = Options()
options.add_argument('--disable-blink-features=AutomationControlled')
options.add_argument("start-maximized")
options.add_argument("--disable-extensions")
driver = webdriver.Chrome(options=options)

I currently have 10 second delays after each action(ie open page, wait, click add to cart, wait, click checkout, wait)

I have implemented a random useragent to be used on each run:

import fake_useragent
ua = UserAgent()
userAgent = ua.random
options.add_argument(f'user-agent={userAgent}')

I have also modified my chromedriver binary as per the comments in THIS THREAD

Error seen when proceeding to order review page: Error seen when proceeding to order review page

like image 678
Bajan Avatar asked Apr 07 '21 15:04

Bajan


1 Answers

After much testing the last few days, here are the options that have allowed me to bypass the restrictions I was facing.

  1. Modified cdc_ string in my chromedriver

  2. Chromedriver options:

     options.add_argument('--disable-blink-features=AutomationControlled')
     options.add_argument("--disable-extensions")
     options.add_experimental_option('useAutomationExtension', False)
     options.add_experimental_option("excludeSwitches", ["enable-automation"])
     chrome_driver = webdriver.Chrome(options=options)
    
  3. Change the property value of the navigator for webdriver to undefined:

chrome_driver.execute_script("Object.defineProperty(navigator, 'webdriver', {get: () => undefined})")

After all three of these were implemented I no longer faced any 403 error when navigating the site and the cart/checkout process.

like image 156
Bajan Avatar answered Sep 28 '22 20:09

Bajan