How can get all results from Youtube API (search API) response

Question

Like the title. I do a request like this :

https://www.googleapis.com/youtube/v3/search?key=AIzaSyDuxczhyyvHWfxKuF3ygW9p0GWmKlvWLYc&part=id,snippet&publishedAfter=2014-12-09T00:00:00Z&publishedBefore=2014-12-11T00:00:00Z&videoCategoryId=GCSG93LXRvICYgRElZ&type=video&maxResults=50&pageToken=

Total result is 1000000. But I just can get 500 results maximum (10 page, 50 results/page). At 10th page, I don't see nextPageToken property to go to the next page. ??? I don't know why. How can I get all of result.

Answer 1

YouTube imposes a soft limit of about 500. There is no direct way to get more than that through the API.

Full details: https://code.google.com/p/gdata-issues/issues/detail?id=4282

Relevant Excerpt:

"We can't provide more than ~500 search results for any arbitrary YouTube query via the API without the quality of the search results severely degrading (duplicates, etc.).

The v1/v2 GData API was updated back in November to limit the number of search results returned to 500. If you specify a start-index of 500 or more, you won't get back any results.

This was supposed to have also gone into effect for the v3 API (which uses a different method of paging through results) but it apparently was not pushed out, so it is still possible to retrieve up to 1000 search results in v3—the last 500 of which are usually of bad quality.

The change to limit v3 to 500 search results will be pushed out sometime in the near future. There will no longer be nextPageTokens returned once you hit 500 results.

I understand that the totalResults that are returned is much higher than 500 in all of these cases, but that is not the same thing as saying that we can effectively return all X million possible results. It's meant as an estimate of the total size of the set of videos that match a query and normally isn't very useful."

Updated - How to get around the 500 result max soft limit

Use the filters 'publishedAfter' and 'publishedBefore' to break up your query into loops of queries by day/week/month until no more results are returned. Each periodic query should return less than 500 results each, but you'll get them all.