I'm in a loop, navigating paged search results. The next button never disappears. I want to keep clicking it until the URL no longer changes. I imagine I will need a requests session? I want to do something like this:
new_url = soup.find("blahblah")['href']
if session.current_url == new_url:
return
Definition and Usage. The get() method sends a GET request to the specified url.
Python requests reading a web pageThe get method issues a GET request; it fetches documents identified by the given URL. The script grabs the content of the www.webcode.me web page. The get method returns a response object. The text attribute contains the content of the response, in Unicode.
On the response object, there is a .url
property that you can use to access the URL of the current page.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With