From the url that is in the code, I am ultimately trying to gather all of the players names from the page. However, when I am using .findAll in order to get all of the list elements, I am yet to be successful. Please advise.
from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup
players_url = 'https://stats.nba.com/players/list/?Historic=Y'
# Opening up the Connection and grabbing the page
uClient = uReq(players_url)
page_html = uClient.read()
players_soup = soup(page_html, "html.parser")
# Taking all of the elements from the unordered lists that contains all of the players.
list_elements = players_soup.findAll('li', {'class': 'players-list__name'})
As @Oluwafemi Sule suggested it is better to use selenium
together with BS
:
from bs4 import BeautifulSoup
from selenium import webdriver
driver = webdriver.Firefox()
driver.get('https://stats.nba.com/players/list/?Historic=Y')
soup = BeautifulSoup(driver.page_source, 'lxml')
for div in soup.findAll('li', {'class': 'players-list__name'}):
print(div.find('a').contents[0])
Output:
Abdelnaby, Alaa
Abdul-Aziz, Zaid
Abdul-Jabbar, Kareem
Abdul-Rauf, Mahmoud
Abdul-Wahad, Tariq
etc.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With