Python urllib urlopen not working

Question

I am just trying to fetch data from a live web by using the urllib module, so I wrote a simple example

Here is my code:

import urllib

sock = urllib.request.urlopen("http://diveintopython.org/") 
htmlSource = sock.read()                            
sock.close()                                        
print (htmlSource)  

But I got error like:

Traceback (most recent call last):
  File "D:\test.py", line 3, in <module>
    sock = urllib.request.urlopen("http://diveintopython.org/") 
AttributeError: 'module' object has no attribute 'request'

Answer 1

You are reading the wrong documentation or the wrong Python interpreter version. You tried to use the Python 3 library in Python 2.

Use:

import urllib2

sock = urllib2.urlopen("http://diveintopython.org/") 
htmlSource = sock.read()                            
sock.close()                                        
print htmlSource

The Python 2 urllib2 library was replaced by urllib.request in Python 3.

Answer 2

In Python3 you can use urllib or urllib3

urllib:

import urllib.request
with urllib.request.urlopen('http://docs.python.org') as response:
    htmlSource = response.read()

urllib3:

import urllib3
http = urllib3.PoolManager()
r = http.request('GET', 'http://docs.python.org')
htmlSource = r.data

More details could be found in the urllib or python documentation.