Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Again urllib.error.HTTPError: HTTP Error 400: Bad Request

Hy! I tried to open web-page, that is normally opening in browser, but python just swears and does not want to work.

import urllib.request, urllib.error
f = urllib.request.urlopen('http://www.booking.com/reviewlist.html?cc1=tr;pagename=sapphire')

And another way

import urllib.request, urllib.error
opener=urllib.request.build_opener()
f=opener.open('http://www.booking.com/reviewlist.html?cc1=tr;pagename=sapphi
re')

Both options give one type of error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python34\lib\urllib\request.py", line 461, in open
    response = meth(req, response)
  File "C:\Python34\lib\urllib\request.py", line 571, in http_response
    'http', request, response, code, msg, hdrs)
  File "C:\Python34\lib\urllib\request.py", line 493, in error
    result = self._call_chain(*args)
  File "C:\Python34\lib\urllib\request.py", line 433, in _call_chain
    result = func(*args)
  File "C:\Python34\lib\urllib\request.py", line 676, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "C:\Python34\lib\urllib\request.py", line 461, in open
    response = meth(req, response)
  File "C:\Python34\lib\urllib\request.py", line 571, in http_response
    'http', request, response, code, msg, hdrs)
  File "C:\Python34\lib\urllib\request.py", line 499, in error
    return self._call_chain(*args)
  File "C:\Python34\lib\urllib\request.py", line 433, in _call_chain
    result = func(*args)
  File "C:\Python34\lib\urllib\request.py", line 579, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 400: Bad Request

Any ideas?

like image 383
Wanu Avatar asked Nov 12 '14 18:11

Wanu


People also ask

What does 400 Bad request mean?

The HyperText Transfer Protocol (HTTP) 400 Bad Request response status code indicates that the server cannot or will not process the request due to something that is perceived to be a client error (for example, malformed request syntax, invalid request message framing, or deceptive request routing).

How do I import urllib2 into Python?

import urllib2 response = urllib2. urlopen('https://www.pythonforbeginners.com/') print response.info() html = response. read() # do something response. close() # best practice to close the file Note: you can also use an URL starting with "ftp:", "file:", etc.).


1 Answers

They are probably blocking the fact that it isn't coming from a browser. You probably need a valid User-Agent header or something.

Using requests, this works:

import requests
headers = 
{
 'User-Agent': 'Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)     Chrome/37.0.2049.0 Safari/537.36'
}

r = requests.get('http://www.booking.com/reviewlist.html?cc1=tr;pagename=sapphire', headers=headers)
print r
print r.headers
like image 136
kelsmj Avatar answered Sep 18 '22 20:09

kelsmj