Fetch all pages using python request

Question

I am using an api to fetch orders from a website. The problem is at one time it only fetch only 20 orders. I figured out i need to use a pagination iterator but dont know to use it. How to fetch all the orders all at once.

My code:

def search_orders(self):
    headers = {'Authorization':'Bearer %s' % self.token,'Content-Type':'application/json',}
    url = "https://api.flipkart.net/sellers/orders/search"
    filter = {"filter": {"states": ["APPROVED","PACKED"],},}
    return requests.post(url, data=json.dumps(filter), headers=headers)

Here is a link to documentation.

Documentation

Answer 1

You need to do what the documentation suggests -

The first call to the Search API returns a finite number of results based on the pageSize value. Calling the URL returned in the nextPageURL field of the response gets the subsequent pages of the search result.

nextPageUrl - String - A GET call on this URL fetches the next page results. Not present for the last page

(Emphasis mine)

You can use response.json() to get the json of the response. Then you can check the flag - hasMore - to see if there are more if so, use requests.get() to get the response for next page, and keep doing this till hasMore is false. Example -

def search_orders(self):
    headers = {'Authorization':'Bearer %s' % self.token,'Content-Type':'application/json',}
    url = "https://api.flipkart.net/sellers/orders/search"
    filter = {"filter": {"states": ["APPROVED","PACKED"],},}
    s = requests.Session()
    response = s.post(url, data=json.dumps(filter), headers=headers)
    orderList = []
    resp_json = response.json()
    orderList.append(resp_json["orderItems"])
    while resp_json.get('hasMore') == True:
        response = s.get('"https://api.flipkart.net/sellers{0}'.format(resp_json['nextPageUrl']))
        resp_json = response.json()
        orderList.append(resp_json["orderItems"])
    return orderList

The above code should return the complete list of orders.