how to click button in headless and button have div tag inside it using selenium python?

Question

using this code for without headless approch.. website link: https://www.na-kd.com/en/sweaters?sortBy=popularity&count=108

try:
     element = self.driver.find_element_by_xpath('//*[@id="container"]/div/div/div[3]/div/div[4]/div/div[1]/div[2]/div[1]/button')
     self.driver.execute_script("arguments[0].click();", element)

except Exception as e:
     print('Error in clicking BTN : '+str(e))

Because this btn have div-tag inside it so it is not working with headless and virtual-display.

I also Try wait:

    try:
        element=WebDriverWait(self.driver, 20).until(
            EC.element_to_be_clickable((By.XPATH, '//*[@id="container"]/div/div/div[3]/div/div[4]/div/div[1]/div[2]/div[1]/button')))
        self.driver.execute_script("arguments[0].click();", element)

    except Exception as e:
        print('Error in clicking BTN : '+str(e))

chromedriver --version
ChromeDriver 78.0.3904.70
Google Chrome 78.0.3904.108

Answer 1

Using the headless mode when you fire any event add window-size() Because headless browser can't recognise where to click without window size.

To click on Load more products button Induce WebDriverWait() and wait for element_to_be_clickable() and use below xpath

To verify this whether button is clicked or not just scroll down the page and get the value from div tag.

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
import time

# Headless option with window-size()
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('window-size=1920x1080');

driver = webdriver.Chrome(chrome_options=chrome_options)
driver.get("https://www.na-kd.com/en/sweaters?sortBy=popularity&count=108&ssr=on&loadfailure=1")

# Load more products button  

element=WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//button[.//div[text()='Load more products']]")))
driver.execute_script("arguments[0].click();", element)

# To verify that whether button is clicked or not
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
time.sleep(2) #wait for page to load
print(WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//div[@class='qa6 qmz qn0']"))).text)