DEV Community

Cover image for Vet a YouTube creator before you sponsor them: comments per view, reply rate and comment quality
Joshua Smith
Joshua Smith

Posted on

Vet a YouTube creator before you sponsor them: comments per view, reply rate and comment quality

#youtube #marketing #python #influencermarketing

A sponsorship is paid for the audience, and the audience is the part a media kit tells you least about. Subscriber counts can be bought, view counts can come from one viral video, and a comment section can be padded with "great video!" from accounts that never watched. This post checks a shortlist of creators with public data before you sign: engagement per view, how often the creator talks to viewers, and how much of the comment section is real conversation.

It starts where the influencer shortlist post in this series ends: you already have a handful of channels that look right on size and topic.

What to measure, and why

From the videos:

  • Median views on recent uploads ÷ subscribers. Use the median of the last 10-15 uploads, not the average, so one viral video does not carry the number. A large channel whose recent videos reach a small fraction of its subscribers has an audience that has moved on.
  • Comments and likes per 1,000 views. Engagement relative to reach, so channels of different sizes compare fairly.
  • Upload frequency. A channel that posts twice a month will not deliver a weekly integration.

From the comments:

  • Creator reply rate. The share of top comments where the creator replied. Creators who talk to their viewers usually have viewers who listen to them, including during a sponsor segment.
  • Generic share. Comments of one or two words, emoji-only comments, and the same text posted by different accounts. Some of that is normal. A comment section that is mostly that is a warning sign.
  • Repeat-commenter concentration. The share of comments written by the ten most active commenters. A small group commenting on everything can be a loyal community or an engagement group; read their comments to tell which.

None of these has a universal "good" threshold. Compare creators against each other, in the same niche and size band.

Step 1: video statistics

Best Damn YouTube Scraper returns each video's views, likes and comment count, plus the channel's subscriber count, through the official YouTube Data API. Give it the channels and take the last 15 uploads:

{
    "startUrls": ["https://www.youtube.com/@CreatorOne", "https://www.youtube.com/@CreatorTwo"],
    "maxResults": 15
}
Enter fullscreen mode Exit fullscreen mode

Step 2: a comment sample with replies

Best Damn YouTube Comments Scraper reads the comments. You need replies on to see whether the creator answered, and isByVideoOwner marks the creator's own comments and replies.

{
    "startUrls": ["https://www.youtube.com/watch?v=RECENT_VIDEO_1", "https://www.youtube.com/watch?v=RECENT_VIDEO_2"],
    "sortBy": "relevance",
    "includeReplies": true,
    "maxComments": 500,
    "authorData": "hash"
}
Enter fullscreen mode Exit fullscreen mode

Replies count toward maxComments, and popular comments can have hundreds of replies. In a test with a 300-comment cap, one video returned just 20 top-level comments; the rest were replies. The script below therefore only judges threads it received in full: it compares the replies delivered with the comment's replyCount. If too few threads are complete, raise maxComments. authorData: "hash" swaps commenter IDs for stable pseudonyms, enough to count repeat commenters without collecting who they are.

Step 3: the scorecard 

import re
from collections import Counter, defaultdict
from datetime import datetime
from statistics import median
from apify_client import ApifyClient

client = ApifyClient("<YOUR_API_TOKEN>")
CHANNELS = ["https://www.youtube.com/@CreatorOne", "https://www.youtube.com/@CreatorTwo"]
SAMPLE_VIDEOS = 5  # per channel, for the comment checks

run = client.actor("josh99smith/youtube-scraper").call(run_input={"startUrls": CHANNELS, "maxResults": 15})
videos = defaultdict(list)
for v in client.dataset(run["defaultDatasetId"]).iterate_items():
    if v["success"]:
        videos[v["channelId"]].append(v)

sample = [v["url"] for vs in videos.values()
          for v in sorted(vs, key=lambda v: v["publishedAt"], reverse=True)[:SAMPLE_VIDEOS]]
run = client.actor("josh99smith/youtube-comments-scraper").call(run_input={
    "startUrls": sample, "sortBy": "relevance", "includeReplies": True,
    "maxComments": 500, "authorData": "hash"})
comments = defaultdict(list)
for c in client.dataset(run["defaultDatasetId"]).iterate_items():
    if c["success"]:
        comments[c["videoChannelId"]].append(c)

def is_generic(text):
    t = text.strip()
    return len(re.findall(r"\w+", t)) <= 2  # covers emoji-only and "first!" too

def per_1k(part, whole):
    return round(1000 * part / whole, 1) if whole else None

for channel_id, vs in videos.items():
    subs = vs[0]["channelSubscribers"]
    views = [v["viewCount"] or 0 for v in vs]
    dates = sorted(datetime.fromisoformat(v["publishedAt"].replace("Z", "+00:00")) for v in vs)
    months = max((dates[-1] - dates[0]).days / 30.4, 1)

    cs = comments[channel_id]
    tops = {c["commentId"]: c for c in cs if not c["isReply"]}
    replies = defaultdict(list)
    for c in cs:
        if c["isReply"]:
            replies[c["parentCommentId"]].append(c)
    complete = [t for t in tops.values() if len(replies[t["commentId"]]) >= (t["replyCount"] or 0)]
    answered = [t for t in complete if any(r["isByVideoOwner"] for r in replies[t["commentId"]])]

    audience = [t for t in tops.values() if not t["isByVideoOwner"]]
    texts = Counter(re.sub(r"\W+", " ", t["text"].lower()).strip() for t in audience)
    copied = sum(n for txt, n in texts.items() if n > 1 and len(txt) > 10)
    generic = sum(is_generic(t["text"]) for t in audience) + copied
    authors = Counter(t["authorChannelId"] for t in audience)
    top10 = sum(n for _, n in authors.most_common(10))

    print(f'\n{vs[0]["channelTitle"]}: {subs:,} subscribers, {len(vs)} recent videos')
    print(f"  median views / subscribers: {median(views) / subs:.2f}" if subs else "  subscribers hidden")
    print(f'  comments per 1k views: {per_1k(sum(v["commentCount"] or 0 for v in vs), sum(views))}')
    print(f'  likes per 1k views:    {per_1k(sum(v["likeCount"] or 0 for v in vs), sum(views))}')
    print(f"  uploads per month:     {len(vs) / months:.1f}")
    print(f"  creator reply rate:    {len(answered)}/{len(complete)} complete threads")
    print(f"  generic or copied:     {generic}/{len(audience)} top comments")
    print(f"  top 10 commenters:     {top10}/{len(audience)} top comments")
Enter fullscreen mode Exit fullscreen mode

Some details that matter:

  • Hidden subscriber counts. Creators can hide them, and the script prints that instead of dividing by zero. Hidden likes or comments show up as null and count as zero here; check the video if a number looks impossible.
  • The reply rate covers top comments only. The script reads YouTube's "Top comments" order, which is where a creator's replies are most visible. A creator who answers late questions on old videos will look less engaged than they are.
  • Copied comments are counted when the same text of more than ten characters appears more than once. That catches pasted promotions and bot rings, and also the occasional popular meme. Read a few before you conclude anything.

Read the comments, not just the numbers

Open ten top comments per creator. You are looking for comments that refer to something specific in the video, viewers answering each other, and the creator's tone when they reply. A channel with ordinary numbers and a comment section where people talk about the content is usually a better buy than one with great numbers and nothing but emoji.

What this cannot tell you: audience country, age, or how many viewers watched the sponsor segment. Those are in the creator's YouTube Studio analytics, which only the creator can see. Ask for a screenshot of the audience tab and recent retention graphs. The checks here tell you whether that screenshot is plausible.

Cost and limits

  • Price. Videos cost $0.001 each, so 15 videos for each of 10 creators is $0.15. Comments cost $0.0004 each; 5 sample videos for each of 10 creators, capped at 500 comments, is at most 25,000 comments, so at most $10. Lower maxComments or SAMPLE_VIDEOS for a first pass. Failures, comments-off videos and quota errors are free, and a run stops at the maximum cost you set.
  • Quota. Both Actors use a built-in API key by default whose 10,000-unit daily quota is shared by all users. Reading videos and comments is cheap in quota (about one unit per 50 videos, one unit per page of 100 comments). If you vet creators every week, a free key of your own from the Google Cloud Console, passed as apiKey, keeps you independent of other users. When a quota runs out, runs stop cleanly and the rest come back as free rate-limited records.
  • Public data only. No login, no Studio analytics, nothing members-only.

Comments are personal data under GDPR and similar laws. Keep authorData on hash, judge the comment section in aggregate, and don't contact or profile individual commenters.

If this saves you time, a review on the Store page helps other people find it, and a star on GitHub helps too.

Disclosure: I built both Actors. Source: github.com/josh99smith/youtube-comments-scraper and github.com/josh99smith/youtube-scraper. YouTube is a trademark of Google; the Actors are not affiliated with Google.

Top comments (0)