Clicking a button with Selenium in Python

Question

Goal: use Selenium and Python to search for company name on LinkedIn's search bar THEN click on the "Companies" button in the navigation to arrive to information about companies that are similar to the keyword (rather than individuals at that company). See below for an example. "CalSTRS" is the company I search for in the search bar. Then I want to click on the "Companies" navigation button.

My Helper Functions: I have defined the following helper functions (including here for reproducibility).

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
from random import randint
from selenium.webdriver.common.action_chains import ActionChains
browser = webdriver.Chrome()

def li_bot_login(usrnm, pwrd):
    ##-----Log into linkedin and get to your feed-----
    browser.get('https://www.linkedin.com')

    ##-----Find the Search Bar-----
    u = browser.find_element_by_name('session_key')
    ##-----Enter Username and Password, Enter-----
    u.send_keys(usrnm)
    p = browser.find_element_by_name('session_password')
    p.send_keys(pwrd + Keys.ENTER)

def li_bot_search(search_term):
    #------Search for term in linkedin search box and land you at the search results page------
    search_box = browser.find_element_by_css_selector('artdeco-typeahead-input.ember-view > input')
    search_box.send_keys(str(search_term) + Keys.ENTER)

def li_bot_close():
    ##-----Close the Webdriver-----
    browser.close()

li_bot_login()
li_bot_search('calstrs')
time.sleep(5)
li_bot_close()

Here is the HTML of the "Companies" button element:

<button data-vertical="COMPANIES" data-ember-action="" data-ember-action-7255="7255">
      Companies
    </button>

And the XPath:

//*[@id="ember1202"]/div[5]/div[3]/div[1]/ul/li[5]/button

What I have tried: Admittedly, I am not very experienced with HTML and CSS so I am probably missing something obvious. Clearly, I am not selecting / interacting with the right element. So far, I have tried...

companies_btn = browser.find_element_by_link_text('Companies')
companies_btn.click()

which returns this traceback:

selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"link text","selector":"Companies"}
  (Session info: chrome=63.0.3239.132)
  (Driver info: chromedriver=2.35.528161 (5b82f2d2aae0ca24b877009200ced9065a772e73),platform=Windows NT 10.0.16299 x86_64)

and

companies_btn_xpath = '//*[@id="ember1202"]/div[5]/div[3]/div[1]/ul/li[5]/button'
browser.find_element_by_xpath(companies_btn_xpath).click()

with this traceback...

selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[@id="ember1202"]/div[5]/div[3]/div[1]/ul/li[5]/button"}
  (Session info: chrome=63.0.3239.132)
  (Driver info: chromedriver=2.35.528161 (5b82f2d2aae0ca24b877009200ced9065a772e73),platform=Windows NT 10.0.16299 x86_64)

and

browser.find_element_by_css_selector('#ember1202 > div.application-outlet > div.authentication-outlet > div.neptune-grid.two-column > ul > li:nth-child(5) > button').click()

which returns this traceback...

selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"#ember1202 > div.application-outlet > div.authentication-outlet > div.neptune-grid.two-column > ul > li:nth-child(5) > button"}
  (Session info: chrome=63.0.3239.132)
  (Driver info: chromedriver=2.35.528161 (5b82f2d2aae0ca24b877009200ced9065a772e73),platform=Windows NT 10.0.16299 x86_64)

Answer 1

It seem that you simply used incorrect selectors.

Note that

• @id of div like "ember1002" is dynamic value, so it will be different each time you visit page: "ember1920", "ember1202", etc...

• find_element_by_link_text() can be applied to links only, e.g. <a>Companies</a>, but not buttons

Try to find button by its text content:

browser.find_element_by_xpath('//button[normalize-space()="Companies"]').click()

Answer 2

With capybara-py (which can be used to drive Selenium), this is as easy as:

page.click_button("Companies")

Bonus: This will be resilient against changes in the implementation of the button, e.g., using <input type="submit" />, etc. It will also be resilient in the face of a delay before the button appears, as click_button() will wait for it to be visible and enabled.