Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

BeautifulSoup find_all limited to 50 results?

I'm trying to get the results from a page using BeautifulSoup:

req_url = 'http://www.xscores.com/soccer/livescores/25-02'
request = requests.get(req_url)
content = request.content
soup = BeautifulSoup(content, "html.parser")
scores = soup.find_all('tr', {'style': 'height:18px;'}, limit=None)
print(len(scores))
>50

I read this previous solution: Beautiful Soup findAll doen't find them all and I tried html.parser, lxml and html5lib, but none of them return more than 50 results. Any suggestions?

Thank you

like image 945
StevenH Avatar asked Feb 27 '17 09:02

StevenH


People also ask

What does Find_all return BeautifulSoup?

Beautiful Soup's find_all(~) method returns a list of all the tags or strings that match a particular criteria.

What is the difference between Find_all () and find () in BeautifulSoup?

find is used for returning the result when the searched element is found on the page. find_all is used for returning all the matches after scanning the entire document.

How do I get all the Div in BeautifulSoup?

You should use soup. find_all('div', attrs={'class': None}) the look to the div without any class attribute.

What does Find_all do in Python?

find_all() returns all the tags and strings that match your filters.


2 Answers

Try using css-selector query.

scores = soup.select('#scoretable > tr[style*="height:18px;"]')
print(len(scores))

>>>613
like image 65
Zroq Avatar answered Oct 16 '22 02:10

Zroq


Try this -

req_url = 'http://www.xscores.com/soccer/livescores/25-02'
request = requests.get(req_url)
html=request.text
soup = BeautifulSoup(html, "html5lib")
scoretable=soup.find('tbody',id='scoretable')
scores=scoretable.find_all('tr')
len(scores)
>617
like image 29
Aditya Mishra Avatar answered Oct 16 '22 01:10

Aditya Mishra