Selenium stuck on "Checking your browser before accessing URL"

Question

This is a recent problem, it began I think three or four days ago. It is not isolated to my own system, as I was running the software on a remote server as well (Windows 10, Windows Server). It is not also not isolated to any specific URL, as I can't get past any URL that has this check now.

Title: "Just a moment..." "Checking your browser before accessing URL". "This process is automatic. Your browser will redirect to your requested content shortly." "Please allow up to 5 seconds..." "DDos Protection by Cloudflare" "Ray Id: xxxxxxxxxxxxxxxxxx"

• I've attempted different systems (both windows based)
• I've attempted different drivers (gecko and chrome)
• I've attempted different urls

from selenium import webdriver
driver = webdriver.Chrome()
driver.get('wwww.etherdelta.com')

Does anyone know how I can resolve this; or is it time to put poor ol' timmy (the program) down?

Answer 1

I had the same issue with firefox. I was able to solve it by switching to Chrome.
Example code:

from selenium import webdriver
url = "<WEBSITE>"
options = webdriver.ChromeOptions()
options.add_argument("--disable-blink-features=AutomationControlled")
driver = webdriver.Chrome(options=options)
driver.get(url)

"--disable-blink-features=AutomationControlled" hides the "navigator.webdriver" flag.
See Selenium webdriver: Modifying navigator.webdriver flag to prevent selenium detection

Edit

You also have to change some default variables of chromedriver.
Example with perl:

perl -pi -e 's/cdc_/dog_/g' /path/to/chromedriver

For more details look at the original post.
See Can a website detect when you are using selenium with chromedriver?

Edit 2

Cloudflare keeps adjusting their algorithm so you could try to use undetected-chromedriver instead of the manual changing of the chromedriver.

undetected-chromedriver is an optimized Selenium Chromedriver patch which should not trigger anti-bot services. It automatically downloads the driver binary and patches it.

Wether this will work or not kinda depends on the website and the current state of the development. Cloudflare seems to track the development of undetected-chromedriver.

import undetected_chromedriver as uc
url = "<WEBSITE>"
driver= uc.Chrome()
driver.get(url)

Answer 2

Try using yout chrome data folder

from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from webdriver_manager.utils import ChromeType

# Configure browser
options = webdriver.ChromeOptions()
options.add_argument(f"--user-data-dir=C:\\Users\\daria\\AppData\\Local\\Google\\Chrome\\User Data")
options.add_argument("--disable-blink-features=AutomationControlled")

chromedriver = ChromeDriverManager(chrome_type=ChromeType.GOOGLE, 
                                            log_level='0', 
                                            print_first_line=False).install()
driver = webdriver.Chrome(chromedriver, 
                                options=options,
                                service_log_path=None)

input ("End?")