Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

BeautifulSoup - find table with specified class on Wikipedia page

I am trying to find a table in a Wikipedia page using BeautifulSoup and for some reason I don't get the table. Can anyone tell why I don't get the table?

my code:

import BeautifulSoup
import requests

url='https://en.wikipedia.org/wiki/List_of_National_Historic_Landmarks_in_Louisiana'
r=requests.get(url)
url=r.content
soup = BeautifulSoup(url,'html.parser')

tab=soup.find("table",{"class":"wikitable sortable jquery-tablesorter"})
print tab

prints: None

like image 442
Tom Avatar asked Nov 17 '15 20:11

Tom


1 Answers

You shouldn't use jquery-tablesorter to select against in the response you get from requests because it is dynamically applied after the page loads. If you omit that, you should be good to go.

tab = soup.find("table",{"class":"wikitable sortable"})
like image 108
wpercy Avatar answered Nov 15 '22 21:11

wpercy