YouTube Data API v3 Comment Thread Discrepency

Question

I have been trying to get a list of comments out using the new V3 Data API with mixed results.

For some videos, you only get a subset of comments out. I have noticed this on a few videos, but for this specific case I will use video ID = U55NGD9Jm7M

You can find all the comments on this video in the WebUI here: https://www.youtube.com/all_comments?v=U55NGD9Jm7M

At the time of posting, there were 5,499 comments on this video.

API Results:

When querying https://www.googleapis.com/youtube/v3/commentThreads?part=id,snippet,replies&textFormat=plainText&maxResults=100&videoId=U55NGD9Jm7M&key={YOUR_API_KEY} I am only getting about 317 comments (including Paging, and counting all replies) (sorted chronologically).

Verification Research:

If you select "Top Comments" from the drop down and then scroll down and click "More" over and over again, you get over 1,000 comments (I stopped at about 1,000)

If you then select "Newest First" from the dropdown and repeat the process (more ... more ... more) you will find that there are about 317 comments before you are unable to show any more comments.

I find it quite odd that there is a discrepancy in the UI, but thankful that the API lines up with part of the UI. Has anyone else noticed this? Is there a way to get the full text of all 5,499 comments?

Thanks!

Jason

Follow-up 1

As a follow-up, I was able to isolate one comment using View->Source (Thread ID z12wzfzhtybgz13kj22ocvsz2unrtn1qj04) and fetch all the information from this comment in the API here: https://www.googleapis.com/youtube/v3/commentThreads?part=id%2Csnippet%2Creplies&id=z12wzfzhtybgz13kj22ocvsz2unrtn1qj04&maxResults=100&key={YOUR_API_KEY})

It even mentions the correct VideoID that the comment is associated with. However, when you query by Video, this comment ID is not returned.

Follow-up 2 I refresh the Web UI of the All Comments, and there was a significantly different list of comments that are being returned

Answer 1

The commentsThread.list call can only return a maximum of 100 results (see maxResults in the documentation). If you wanted to get more comment threads, you'd have to pass in the nextPageToken you get from your initial call into a subsequent API call.

For example:

https://www.googleapis.com/youtube/v3/commentThreads?part=snippet&videoId=U55NGD9Jm7M&maxResults=100&key=API_KEY

gives you 100 comment threads, and the nextPageToken is Cg0Qk9fa7fHgxgIgACgBEhQIARCY49LZ5eDGAhi4rNGIrZrGAhgCIGM. If you include that token in a new API call, like so:

https://www.googleapis.com/youtube/v3/commentThreads?part=snippet&videoId=Dlj6SUg9B04&&maxResults=100&nextPageToken=Cg0Qk9fa7fHgxgIgACgBEhQIARCY49LZ5eDGAhi4rNGIrZrGAhgCIGM&key=API_KEY

You get a completely different set of comment threads. You can double check this by specifying order=time in both API calls. You'll see that the earliest comment threads for both calls are different, and you won't find the comment Thread ID for either in the other call's results. To get even more comment threads, you take the nextPageToken from the newer call's results and do the same thing again (until the call doesn't give you another nextPageToken, meaning you're on the last page, and there are no more comment threads to return).