Google Custom Search API start=100 causes error 400

Question

I have a script that uses Google Custom Search API, iterating through multiple results pages.

https://www.googleapis.com/customsearch/v1?key=[[KEY]]&num=10&hl=en&start=0&cx=[[CX]]&q=%22bank%22&sort=date&googlehost=www.google.com

https://www.googleapis.com/customsearch/v1?key=[[KEY]]&num=10&hl=en&start=10&cx=[[CX]]&q=%22bank%22&sort=date&googlehost=www.google.com

https://www.googleapis.com/customsearch/v1?key=[[KEY]]&num=10&hl=en&start=20&cx=[[CX]]&q=%22bank%22&sort=date&googlehost=www.google.com

In all of the above examples, I get proper responses. The query response claims that there are 17,900 results to the search. When the script reaches start=100, however:

https://www.googleapis.com/customsearch/v1?key=[[KEY]]&num=10&hl=en&start=100&cx=[[CX]]&q=%22bank%22&sort=date&googlehost=www.google.com

I receive the following response (this is the JSON response transformed into a PHP object):

stdClass Object (
        [error] => stdClass Object
            (
                [errors] => Array
                    (
                        [0] => stdClass Object
                            (
                                [domain] => global
                                [reason] => invalid
                                [message] => Invalid Value
                            )
                    )
                [code] => 400
                [message] => Invalid Value
            ) )

This is despite the fact that the results I receive in start=90 claim that the next page exists:

"nextPage": [
   {
    "title": "Google Custom Search - \"bank\"",
    "totalResults": "17900",
    "searchTerms": "\"bank\"",
    "count": 10,
    "startIndex": 100,
    "inputEncoding": "utf8",
    "outputEncoding": "utf8",
    "safe": "off",
    "cx": "[[CX VALUE]]",
    "sort": "date",
    "googleHost": "www.google.com",
    "hl": "en"
   }
  ]

Playing around with the API shows that this invalid value error appears exactly when start=92. Furthermore, this is the exact page where this error appears for every keyword search. Any help with the issue would be appreciated. Is this because this is a free version of Google Custom Search?

Answer 1

This info is not at all easy to find, and I found it mentioned officially in only one place on google.com. I found a single line in the Custom Search JSON API docs describing the nextPage response element:

Note: This API returns up to the first 100 results only.

There is no mention of this being limited to the free API only. I also found user reports confirming that the limit applies even if you sign up for billing, eg see this related SO question and the linked blog post.

OTOH the 100 queries/day limit (as opposed to the results limit you are seeing) is well documented, and there is info about how to bypass that everywhere (by signing up for billing, of course).