How to limit download rate of HTTP requests in requests python library?

Question

Is it possible to limit the download rate of GET requests using the requests Python library? For instance, with a command like this:

r = requests.get('https://stackoverflow.com/')

...is it possible to limit the download rate? I'm hoping for something similar to this wget command:

wget --limit-rate=20k https://stackoverflow.com/

I know it's possible with urllib2. I'm asking specifically about the requests library.

Answer 1

There are several approaches to rate limiting; one of them is token bucket, for which you can find a recipe here and another one here.

Usually you would want to do throttling or rate limiting on socket.send() and socket.recv(). You could play with socket-throttle and see if it does what you need.

---

This is not to be confused with x-ratelimit rate limiting response headers, which are related to a number of requests rather than a download / transfer rate.