Return last URL in sequence of redirects

Question

I sometimes need to parse with Beautiful Soup and Requests URLs that are provided as such:

http://bit.ly/sdflksdfwefwe

http://stup.id/sdfslkjsfsd

http://0.r.msn.com/sdflksdflsdj

Of course, these URLs generally 'resolve' to a canonical URL some as http://real-website.com/page.html. How can I get the last URL in the resolution / redirect chain?

My code generally looks like this:

from bs4 import BeautifulSoup
import requests

response = requests.get(url)
soup = bs4.BeautifulSoup(response.text, from_encoding=response.encoding)
canonical_url = response.??? ## This is what I need to know

Note that I don't mean to query http://bit.ly/bllsht to see where it goes, but rather when I am using Beautiful Soup to already parse the page that it returns, to also get the canonical URL that was the last in the redirect chain.

Thanks.

Answer 1

It's in the url attribute of your response object.

>>> response = requests.get('http://bit.ly/bllsht')
>>> response.url
  > u'http://www.thenews.org/sports/well-hey-there-murray-state-1-21-11-1.2436937'

You could easily find this information in the “Quick Start” page.