Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Fetch list of 50,000 most subscribed channels

I'm trying to figure out a way to grab the top 50,000 most subscribed youtube channels using javascript. These only need to be grabbed once and will be stored in a file to be used for an autocomplete input in a webpage.

I've gotten pretty close to getting the first top 50 by using search:list (/youtube/v3/search) by searching with parameters maxResults=50, order=viewCount, part=snippet, type=channel, fields=nextPageToken,items(snippet(channelId,title))

Returning:

{
 "nextPageToken": "CDIQAA",
 "items": [{
   "snippet": {
    "channelId": "UC-9-kyTW8ZkZNDHQJ6FgpwQ",
    "title": "Music"
   }
  },{
   "snippet": {
    "channelId": "UC-lHJZR3Gqxm24_Vd_AJ5Yw",
    "title": "PewDiePie"
   }
  },{
   "snippet": {
    "channelId": "UCVPYbobPRzz0SjinWekjUBw",
    "title": "Анатолий Шарий"
   }
  },{
   "snippet": {
    "channelId": "UCam8T03EOFBsNdR0thrFHdQ",
    "title": "VEGETTA777"
   }
  },...

Then all I'd have to do is fetch that 1000 more times using the nextPageToken to get a list of the top 50,000.

Unfortunately, sorting by relevance, rating, viewCount, or nothing is not yielding the 50 most subscribed channels, and there doesn't seem to be any sort of way to order them by subscriber count according to the documentation; so it seems like i am stuck.

like image 579
Marc Guiselin Avatar asked Apr 21 '17 15:04

Marc Guiselin


1 Answers

Just before you writing your 50 results in file (or database), you can make one more API call, using channelId field from your result, and merge all of them with comma delimited and make another API call Channels: list.
On that page for example you can use following parameters:
(these are IDs from your example above)

part=statistics
id=UC-9-kyTW8ZkZNDHQJ6FgpwQ,UC-lHJZR3Gqxm24_Vd_AJ5Yw,UCVPYbobPRzz0SjinWekjUBw,UCam8T03EOFBsNdR0thrFHdQ`

And result will look something like this:

{
"kind": "youtube#channel",
"etag": "\"m2yskBQFythfE4irbTIeOgYYfBU/MG6zgnd09mqb3nAdyRnPDgFwfkE\"",
"id": "UC-lHJZR3Gqxm24_Vd_AJ5Yw",
"statistics": {
     "viewCount": "15194203723",
     "commentCount": "289181",
     "subscriberCount": "54913094",
     "hiddenSubscriberCount": false,
     "videoCount": "3175"
    }
}

And you can take subscriberCount from result for each channel.

I know, this is not the way to sort your 50 results while writing into the file, but with this you can sort later your results by "subscriber count" while fetching from file for your autocomplete input.

I didn't find any other way to sort results by subscriber count, so maybe this can be helpful.

like image 150
Nebojsa Nebojsa Avatar answered Oct 16 '22 23:10

Nebojsa Nebojsa