Getting '403 Forbidden' error when using headless chrome with python selenium

Question

When using regualr chromedriver, my tests run without any issues and I get the test results. When I am trying to launch headless chrome with python and selenium, I get a '403 Forbidden' error on the screenshot and selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: on the console. I passed all these arguments and still get the same result :

chrome_options = Options()
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--window-size=1920,1080')
chrome_options.add_argument('--headless')
chrome_options.add_argument('--disable-gpu')
chrome_options.add_argument('--allow-running-insecure-content')
chrome_options.add_argument("--headless")
driver = webdriver.Chrome(chrome_options=chrome_options)

I also tried the from fake_headers import Headers but the issue still persists

Answer 1

I found another solution which is similar to the first answer. The explanation for the issue can be found here. This took care of the issue for me.

chrome_options = Options()
user_agent = 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.50 Safari/537.36'
chrome_options.add_argument(f'user-agent={user_agent}')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--window-size=1920,1080')
chrome_options.add_argument('--headless')
chrome_options.add_argument('--disable-gpu')
chrome_options.add_argument('--allow-running-insecure-content')
chrome_options.add_argument("--headless")
driver = webdriver.Chrome(chrome_options=chrome_options)