Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python requests - 403 forbidden - despite setting `User-Agent` headers

import requests
import webbrowser
from bs4 import BeautifulSoup

url = 'https://www.gamefaqs.com'
#headers={'User-Agent': 'Mozilla/5.0'}    
headers ={'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36'}


response = requests.get(url, headers)

response.status_code is returning 403. I can browse the website using firefox/chrome, so It seems to be a coding error.

I can't figure out what mistake I'm making.

Thank you.

like image 434
Moondra Avatar asked Jul 13 '17 16:07

Moondra


2 Answers

This works if you make the request through a Session object.

import requests

session = requests.Session()
response = session.get('https://www.gamefaqs.com', headers={'User-Agent': 'Mozilla/5.0'})

print(response.status_code)

Output:

200
like image 137
cs95 Avatar answered Sep 23 '22 23:09

cs95


Using keyword argument works for me:

import requests
headers={'User-Agent': 'Mozilla/5.0'}
response = requests.get('https://www.gamefaqs.com', headers=headers)
like image 21
Stephen Avatar answered Sep 23 '22 23:09

Stephen