Python requests Response 504

Question

I'm learning Python, and I'm trying to request access to a website using the command requests. I'm doing the following:

import requests
requests.get("http://www.charitystars.com")

However I get <Response [504]>, which should be an error because the soup command soup = BeautifulSoup(r.content) returns an empty line. I tried with other websites and I get <Response [200]>, and the soup works. So I wonder why the command doesn't work on the first website, and what Response 504 actually means.

Answer 1

This page doesn't like scripts/bots and it checks header user-agent.

It can also need this information to display correct page - different for desktop, tablet, smartfon.

import requests

headers = {'User-Agent': 'Mozilla/5.0'}

r = requests.get("http://www.charitystars.com/", headers=headers)

print(r.status_code)

---

BTW: requests as default uses "User-Agent": "python-requests/2.12.1"

You can use portal http://httpbin.org to see your requests.

import requests

r = requests.get("http://httpbin.org/get")

print(r.text)