How to select a dropdown menu using Selenium Python

Question

I'm working on a personal project, trying to use selenium to web scrape my locals teams results from a website. The site has three drop-down lists, at the moment I'd be happy to just be able to alter one. The code below clicks reject cookies when I enter the page, that's what the WebDriver line does. I'm getting as far as 'dropdown' and then an error that no such element exists. I've tried CSS_SELECTOR, XPATH and all the other options to no avail. I am only new to this so it could be something simple I'm missing but I've read a lot of the previous forums with similar questions and the answers didn't work for me.

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import Select
import time

driver = webdriver.Firefox()
driver.get("https://www.leinsterrugby.ie/domestic-rugby/domestic-fixtures-results/")

time.sleep(3)
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//*[@id='onetrust-reject-all-handler']"))).click()
time.sleep(3)

dropdown=Select(driver.find_element(By.XPATH,value='//*[@id="groups_data"]')
time.sleep(5)
dropdown.select_by_value('5756')

driver.quit() # close browser

Answer 1

Many of those lines can be simplified by using SeleniumBase.

Here's code that searches for results on that site and then prints the results: (self.switch_to_frame() is needed to switch to the iframe, and self.select_option_by_text() is for the dropdowns.)

from seleniumbase import BaseCase

class RugbyResultsTests(BaseCase):
    def test_get_rubgy_results(self):
        self.open("https://www.leinsterrugby.ie/domestic-rugby/domestic-fixtures-results/")
        self.switch_to_frame('iframe[src*="sportsmanager.ie/~leinsterrugby/"]')
        self.sleep(1)
        self.select_option_by_text("select#usercompyear", "2020-2021")
        self.sleep(3)
        self.select_option_by_text("select#groups_data", "Leinster Seconds League (J2)")
        self.sleep(1)
        self.click('a[href*="sportsmanager.ie/~leinsterrugby/league/155189"]')
        self.sleep(2)
        print("\n" + self.get_text("table.table-condensed"))

Output: (After running the above test with pytest with seleniumbase installed via pip.)

POS TEAM PLD W D L PF PA DIFF BP BP L DED PTS
1 Ashbourne 1 1 0 0 18 17 1 0 0 0 4
2 Gorey 1 1 0 0 0 0 0 0 0 0 4
3 Kilkenny 1 0 0 1 17 18 -1 1 1 0 1
4 Suttonians 0 0 0 0 0 0 0 0 0 0 0
5 Monkstown 0 0 0 0 0 0 0 0 0 0 0
6 Seapoint 0 0 0 0 0 0 0 0 0 0 0
7 Bective Rangers 0 0 0 0 0 0 0 0 0 0 0
8 Dundalk 1 0 0 1 0 0 0 0 0 0 0

The default browser for SeleniumBase tests is Chrome, but you can change that by calling pytest --firefox.