Reading a 40,000-comment YouTube thread with the Data API

tian song — Fri, 09 Oct 2026 06:19:23 +0000

Most tutorials that show you how to read YouTube comments stop at the
first page. Here is what actually happens when you try to read a thread
that has 40,000 of them.

The endpoint, and what it actually costs

Comments live behind commentThreads.list on the YouTube Data API v3:

GET https://www.googleapis.com/youtube/v3/commentThreads
    ?part=snippet,replies
    &videoId=VIDEO_ID
    &maxResults=100
    &pageToken=NEXT_PAGE
    &key=YOUR_API_KEY

Two things bite people here.

It is quota-metered, not rate-limited. commentThreads.list costs 1 unit
per call against a default 10,000-unit daily quota. At maxResults=100 that
is 100 calls — and 100 calls is 1% of your day. A busy video with 40,000
comments needs 400 calls. You will hit the ceiling faster than you expect,
and the error is a plain 403 quotaExceeded, not a retryable 429.

maxResults is capped at 100 and the cap is silent. Ask for 500 and you
get 100 with no warning. Loop until a page returns fewer items than you asked
for, or you will spin forever on a video whose comment count is a multiple
of 100.

Pagination that actually terminates

The naive loop breaks in three places: the last page, a video with comments
disabled, and a video that is age-restricted.

import requests

def read_thread(video_id: str, api_key: str):
    url = "https://www.googleapis.com/youtube/v3/commentThreads"
    token, seen = None, 0

    while True:
        params = {
            "part": "snippet,replies",
            "videoId": video_id,
            "maxResults": 100,
            "key": api_key,
        }
        if token:
            params["pageToken"] = token

        resp = requests.get(url, params=params, timeout=15)

        # 403 covers both "comments disabled" and "quota exhausted".
        # Read the reason, do not just retry.
        if resp.status_code == 403:
            reason = resp.json()["error"]["errors"][0]["reason"]
            if reason == "commentsDisabled":
                return seen
            raise RuntimeError(f"quota or permission problem: {reason}")

        payload = resp.json()
        items = payload.get("items", [])
        seen += len(items)

        token = payload.get("nextPageToken")
        if not token:
            return seen

The nextPageToken is absent on the final page — that is your exit
condition. Do not compare against pageInfo.totalResults; it is an estimate
and it drifts on large threads.

Replies are a second request

commentThreads.list returns the top-level comment plus, by default, a
single reply. Getting the rest means calling comments.list with
parentId. On a thread with heavy back-and-forth this doubles or triples
your quota spend, and there is no way to know in advance how much.

If you only need the top-level conversation, do not request replies at all —
you get roughly 3x the coverage for the same quota.

When you do not want to write any of this

I built Apriocity because I kept writing this same
pagination loop for one-off research tasks. It does the paging, the quota
accounting and the reply expansion for you, and hands back CSV or JSON:

paste a video link, read the whole thread, search it by keyword
export the slice as CSV or JSON
batch mode: up to ten videos, one ZIP back

It uses the same public API described above, so what you see is exactly
what is publicly visible — nothing is stored on the server side.

The short version

commentThreads.list = 1 quota unit per call, maxResults capped at 100
nextPageToken missing = done; do not trust totalResults
403 is overloaded: check the reason field before retrying
Replies cost extra calls — skip them unless you need them