Python/Beautiful soup find_all() doesn't find all

Question

When I use find_all() I should get 100 results but I only get 25.

CODE

Here is the code where i crawl tweakers and try to return each element where the class is equal to largethumb.

Once i've done that I filter out the name and price.

my_url = 'https://tweakers.net/categorie/545/geheugen-intern/producten/'
uReq(my_url)
uClient = uReq(my_url)
page_html = uClient.read()
uClient.close()
page_soup = soup(page_html, "html.parser")

#should be 100 tr object's
products = page_soup.find_all("tr", attrs={"class": "largethumb"})
for product in products:
    title = product.p.text
    price_container = product.find_all("p", {"class": "price"})
    price = price_container[0].text
    lijst = title, price
    print(lijst)

RESULT

The result is 25 times this.

('Corsair Vengeance LPX CMK16GX4M2B3000C15', '€ 174,90')

Answer 1

The website in question by default displays 25 search results. If it is different in your web browser, that's because of cookies your browser has from the site in question. If you want to get 100 results, edit my_url like so:

my_url = 'https://tweakers.net/categorie/545/geheugen-intern/producten/?pageSize=100&page=1'
uReq(my_url)
uClient = uReq(my_url)
page_html = uClient.read()
uClient.close()
page_soup = soup(page_html, "html.parser")

# WILL be 100 tr object's
products = page_soup.find_all("tr", attrs={"class": "largethumb"})
for product in products:
    title = product.p.text
    price_container = product.find_all("p", {"class": "price"})
    price = price_container[0].text
    lijst = title, price
    print(lijst)

Proof it works:

>>> from requests import get
>>> from bs4 import BeautifulSoup
>>> my_url = 'https://tweakers.net/categorie/545/geheugen-
intern/producten/?pageSize=100&page=1'
>>> r = get(my_url)
>>> soup = BeautifulSoup(r.content, 'html5lib')
>>> len(soup.find_all('tr', attrs={"class": "largethumb"}))
100

If you hover over the 100 results button in the bottom left hand corner, you see this url is what they redirect you to. Happy scraping!