Does requests.get() (Python, Requests module) pause a script until the response arrives?

Question

This question is regarding the Requests module for Python.

I'm using requests.get() and running into a problem where I exceed the rate limit of the requests the server I'm contacting allows me. I have set a time.sleep() after each requests.get() to something that is an order of magnitude larger. The code is like this:

        for url in url_list:
            success = False
            response = requests.get(url) 
            while not success:
                time.sleep(1/250)
                if str(response) == "<Response [200]>":
                    time.sleep(1/250) # Wait Xs between API call      
                    do_stuff(response.text)
                    success = True

                else:
                    print(response)
                    print(response.headers)
                    time.sleep(3)
                    response = requests.get(url)

I am constantly running up my rate here which is 300 requests / second, where as I should be sending only 125 requests per second at most. Does Requests not pause the script until the response arrives? I'm not sure. How can I make sure I don't send more requests than my rate?

Answer 1

The requests.get function is a blocking call. It will wait until the response arrives before the rest of your program will execute. If you want to be able to do other things, you will probably want to look at the asyncio or multiprocessing modules.