get more than 10 results Google custom search API

Question

I am trying to use Google custom search API,what I want to do is search the first 20 results, I tried changing the num=10 in URL to 20 but gives 400 Error, How can I fix or requests the second page of results ?(Note I am using search entire web)

Here is the code I am using

import requests,json
url="https://www.googleapis.com/customsearch/v1?q=SmartyKat+Catnip+Cat+Toys&cx=012572433248785697579%3A1mazi7ctlvm&num=10&fields=items(link%2Cpagemap%2Ctitle)&key={YOUR_API_KEY}"
res=requests.get(url)
di=json.loads(res.text)

Answer 1

Unfortunately, it is not possible to receive more than 10 results from Google custom search API. However, if you do want more results you can make multiple calls by increasing your start parameter by 10.

See this link: https://developers.google.com/custom-search/v1/using_rest#query-params

Answer 2

The information in the accepted answer https://stackoverflow.com/a/55866268/42346 is accurate.

Below is a Python function I wrote as an extension of the function in the 4th step of this answer https://stackoverflow.com/a/37084643/42346 to return up to 100 results from the Google Search API. It increases the start parameter by 10 for each API call, handling the number of results to return automatically. For example, if you request 25 results the function will induce 3 API calls of: 10 results, 10 results, and 5 results.

Background information:
For instructions on how to set-up a Google Custom Search engine: https://stackoverflow.com/a/37084643/42346
More detail about how to specify that it search the entire web here: https://stackoverflow.com/a/11206266/42346

from googleapiclient.discovery import build
from pprint import pprint as pp
import math

def google_search(search_term, api_key, cse_id, **kwargs):
    service = build("customsearch", "v1", developerKey=api_key)
    
    num_search_results = kwargs['num']
    if num_search_results > 100:
        raise NotImplementedError('Google Custom Search API supports max of 100 results')
    elif num_search_results > 10:
        kwargs['num'] = 10 # this cannot be > 10 in API call 
        calls_to_make = math.ceil(num_search_results / 10)
    else:
        calls_to_make = 1
        
    kwargs['start'] = start_item = 1
    items_to_return = []
    while calls_to_make > 0:
        res = service.cse().list(q=search_term, cx=cse_id, **kwargs).execute()
        items_to_return.extend(res['items'])
        calls_to_make -= 1
        start_item += 10
        kwargs['start'] = start_item
        leftover = num_search_results - start_item + 1
        if 0 < leftover < 10:
            kwargs['num'] = leftover
        
    return items_to_return 

And here's an example of how you'd call that:

NUM_RESULTS = 25
MY_SEARCH = 'why do cats chase their own tails'
MY_API_KEY = 'Google API key'
MY_CSE_ID = 'Custom Search Engine ID'

results = google_search(MY_SEARCH, MY_API_KEY, MY_CSE_ID, num=NUM_RESULTS)
    
for result in results:
    pp(result)