Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What are the differences between the urllib, urllib2, urllib3 and requests module?

In Python, what are the differences between the urllib, urllib2, urllib3 and requests modules? Why are there three? They seem to do the same thing...

like image 323
Paul Biggar Avatar asked Jan 07 '10 03:01

Paul Biggar


People also ask

What is the difference between Urllib and requests?

2) urllib provides the urlencode method which is used for the generation of GET query strings, urllib2 doesn't have such a function. This is one of the reasons why urllib is often used along with urllib2. Requests - Requests' is a simple, easy-to-use HTTP library written in Python.

Is requests faster than Urllib?

I found that time took to send the data from the client to the server took same time for both modules (urllib, requests) but the time it took to return data from the server to the client is more then twice faster in urllib compare to request.

What is the use of Httplib and Urllib modules?

httplib and httplib2 handles HTTP/HTTPs request and response directly and give you more space to do your own job. urllib and urllib2 are build upon httplib, they are more abstract and powerful, but sometimes won't fulfill your specified need about some HTTP related operations.

What would you use Urllib request for?

The urllib. request module defines functions and classes which help in opening URLs (mostly HTTP) in a complex world — basic and digest authentication, redirections, cookies and more. The Requests package is recommended for a higher-level HTTP client interface.


1 Answers

I know it's been said already, but I'd highly recommend the requests Python package.

If you've used languages other than python, you're probably thinking urllib and urllib2 are easy to use, not much code, and highly capable, that's how I used to think. But the requests package is so unbelievably useful and short that everyone should be using it.

First, it supports a fully restful API, and is as easy as:

import requests  resp = requests.get('http://www.mywebsite.com/user') resp = requests.post('http://www.mywebsite.com/user') resp = requests.put('http://www.mywebsite.com/user/put') resp = requests.delete('http://www.mywebsite.com/user/delete') 

Regardless of whether GET / POST, you never have to encode parameters again, it simply takes a dictionary as an argument and is good to go:

userdata = {"firstname": "John", "lastname": "Doe", "password": "jdoe123"} resp = requests.post('http://www.mywebsite.com/user', data=userdata) 

Plus it even has a built in JSON decoder (again, I know json.loads() isn't a lot more to write, but this sure is convenient):

resp.json() 

Or if your response data is just text, use:

resp.text 

This is just the tip of the iceberg. This is the list of features from the requests site:

  • International Domains and URLs
  • Keep-Alive & Connection Pooling
  • Sessions with Cookie Persistence
  • Browser-style SSL Verification
  • Basic/Digest Authentication
  • Elegant Key/Value Cookies
  • Automatic Decompression
  • Unicode Response Bodies
  • Multipart File Uploads
  • Connection Timeouts
  • .netrc support
  • List item
  • Python 2.7, 3.6—3.9
  • Thread-safe.
like image 133
Hutch Avatar answered Oct 16 '22 10:10

Hutch