Python script with Selenium and Chromedriver in headless mode on CentOS7 runs fine when called manually.
options = webdriver.ChromeOptions()
options.add_argument('headless')
options.add_argument('no-sandbox')
self.driver = webdriver.Chrome(chrome_options=options)
When starting script with crontab however it throws this exception at line 4 (above). Full traceback at bottom.
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally (Driver info: chromedriver=2.38.552522
Cron is setup with crontab -e
* * * * * cd /to/path && /to/path/.virtualenvs/selenium/bin/python /to/path/script.py -t arg1 arg2 > /to/path/log.txt 2>&1
This produced errors like chromedriver couldn't be found. I then added following to crontab -e.
1) Use bash instead of sh, although starting python script manually from sh works fine
2) Specify path to chromedriver
SHELL=/bin/bash
PATH=/usr/local/bin/
I tried different suggestions found on the web like adding --no-sandbox options to chromedriver in my script. All didn't help. Please note that I am using chrome in headless mode, so I think I don't need this export DISPLAY=:0 stuff in cron, or Xvfb libs as it used to be.
Python 3.6.1
Selenium 3.4.3
Chromedriver 2.38.552522
google-chrome-stable 65.0.3325.181
Full traceback
Exception in thread <name>:
Traceback (most recent call last):
File "/usr/lib64/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib64/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/path/to/script.py", line 53, in start
self.site_scrape(test_run)
File "/path/to/script.py", line 65, in site
self.driver = webdriver.Chrome(chrome_options=options)
File "/home/<user>/.virtualenvs/selenium/lib64/python3.6/site-packages/selenium/webdriver/chrome/webdriver.py", line 69, in __init__
desired_capabilities=desired_capabilities)
File "/home/<user>/.virtualenvs/selenium/lib64/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 98, in __init__
self.start_session(desired_capabilities, browser_profile)
File "/home/<user>/.virtualenvs/selenium/lib64/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 188, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/home/<user>/.virtualenvs/selenium/lib64/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 256, in execute
self.error_handler.check_response(response)
File "/home/<user>/.virtualenvs/selenium/lib64/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally
(Driver info: chromedriver=2.38.552522 (437e6fbedfa8762dec75e2c5b3ddb86763dc9dcb),platform=Linux 4.14.12-x86_64-linode92 x86_64)
ChromeOptions options = new ChromeOptions() options. addArgument("headless"); ChromeDriver driver = new ChromeDriver(options); In the above code, the browser is instructed to run in the headless mode using the addArgument() method of the ChromeOptions class provided by the Selenium WebDriver.
In order to instantiate the object of ChromeDriver, you can simply create the object with the help of below command. Webdriver driver = New ChromeDriver(); The main motto of the ChromeDriver is to launch Google Chrome. Without that, it is not possible to execute Selenium test scripts in Google Chrome browser.
By saving chromedriver.exe in the same folder als your Python working directory, there's no need to specify the path.
Finally found the solution. Boy this was bugging me for far too long. Issue was following missing PATH directories: /usr/bin, /usr/sbin in cron. Complete cron looks now like this:
SHELL=/bin/bash
PATH=/usr/local/bin/:/usr/bin:/usr/sbin
* * * * * cd /to/path && /to/path/.virtualenvs/selenium/bin/python /to/path/script.py -t arg1 arg2 > /to/path/log.txt 2>&1
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With