Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to get the current page URL in the requests library?

I'm in a loop, navigating paged search results. The next button never disappears. I want to keep clicking it until the URL no longer changes. I imagine I will need a requests session? I want to do something like this:

new_url = soup.find("blahblah")['href']
if session.current_url == new_url:
    return
like image 445
User Avatar asked Dec 21 '15 16:12

User


People also ask

What does requests get URL do?

Definition and Usage. The get() method sends a GET request to the specified url.

How would you request a webpage using Python import requests?

Python requests reading a web pageThe get method issues a GET request; it fetches documents identified by the given URL. The script grabs the content of the www.webcode.me web page. The get method returns a response object. The text attribute contains the content of the response, in Unicode.


1 Answers

On the response object, there is a .url property that you can use to access the URL of the current page.

like image 99
User Avatar answered Oct 04 '22 16:10

User