Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Fetch all pages using python request

I am using an api to fetch orders from a website. The problem is at one time it only fetch only 20 orders. I figured out i need to use a pagination iterator but dont know to use it. How to fetch all the orders all at once.

My code:

def search_orders(self):
    headers = {'Authorization':'Bearer %s' % self.token,'Content-Type':'application/json',}
    url = "https://api.flipkart.net/sellers/orders/search"
    filter = {"filter": {"states": ["APPROVED","PACKED"],},}
    return requests.post(url, data=json.dumps(filter), headers=headers)

Here is a link to documentation.

Documentation

like image 219
Manish Gupta Avatar asked Aug 08 '15 05:08

Manish Gupta


People also ask

What does requests get () do?

The get() method sends a GET request to the specified url.

What is HTTP request in Python?

The requests module allows you to send HTTP requests using Python. The HTTP request returns a Response Object with all the response data (content, encoding, status, etc).


1 Answers

You need to do what the documentation suggests -

The first call to the Search API returns a finite number of results based on the pageSize value. Calling the URL returned in the nextPageURL field of the response gets the subsequent pages of the search result.

nextPageUrl - String - A GET call on this URL fetches the next page results. Not present for the last page

(Emphasis mine)

You can use response.json() to get the json of the response. Then you can check the flag - hasMore - to see if there are more if so, use requests.get() to get the response for next page, and keep doing this till hasMore is false. Example -

def search_orders(self):
    headers = {'Authorization':'Bearer %s' % self.token,'Content-Type':'application/json',}
    url = "https://api.flipkart.net/sellers/orders/search"
    filter = {"filter": {"states": ["APPROVED","PACKED"],},}
    s = requests.Session()
    response = s.post(url, data=json.dumps(filter), headers=headers)
    orderList = []
    resp_json = response.json()
    orderList.append(resp_json["orderItems"])
    while resp_json.get('hasMore') == True:
        response = s.get('"https://api.flipkart.net/sellers{0}'.format(resp_json['nextPageUrl']))
        resp_json = response.json()
        orderList.append(resp_json["orderItems"])
    return orderList

The above code should return the complete list of orders.

like image 91
Anand S Kumar Avatar answered Nov 11 '22 00:11

Anand S Kumar