DEV Community

Qovia
Qovia

Posted on

Comparing Bilibili and YouTube comments on the same topic with Python

#python #api #datascience #tutorial

I build Qovia, the API used in this post. The method here (search a topic on each platform, pull comments, compare) is generic; Qovia is just where the JSON comes from.

The same topic gets discussed very differently on Bilibili and YouTube: different audiences, different languages, different comment cultures. In this tutorial we'll pick one topic, find a popular video about it on each platform, pull the comments, and compare them side by side in a small pandas table. We won't claim anything about the platforms in general; one topic and a handful of videos is a sample, not a study.

Setup 

pip install requests pandas
export QOVIA_KEY=sk-qovia-...
Enter fullscreen mode Exit fullscreen mode

Get a key from Qovia. Requests authenticate with Authorization: Bearer <key>.

import os
import requests

BASE = "https://api.qovia.ai"
HEADERS = {"Authorization": f"Bearer {os.environ['QOVIA_KEY']}"}
spent = 0.0


def get(path, **params):
    global spent
    r = requests.get(f"{BASE}{path}", headers=HEADERS, params=params, timeout=60)
    r.raise_for_status()
    body = r.json()
    spent += float(body.get("meta", {}).get("charged_usd") or 0)
    return body["data"] or {}
Enter fullscreen mode Exit fullscreen mode

Step 1: find one video per platform

A topic needs a keyword in each platform's main language. I'll use TSMC as the example: 台积电 on Bilibili and TSMC on YouTube.

Bilibili search (GET /v1/raw/bilibili/search) takes keyword, page, page_size and order (totalrank, click, pubdate, dm, stow or scores). Results carry the BV id in extra.source_id:

b = get("/v1/raw/bilibili/search", keyword="台积电", order="click", page_size=10)
for i, v in enumerate(b["items"]):
    print(i, v["metrics"]["plays"], v["metrics"]["comments"], v["extra"]["title"])
bvid = b["items"][0]["extra"]["source_id"]
Enter fullscreen mode Exit fullscreen mode

YouTube search (GET /v1/raw/youtube/search) takes keyword, upload_date (hour, today, week, month, year; the catalog says the default is this month) and a cursor. The video id is also in extra.source_id:

y = get("/v1/raw/youtube/search", keyword="TSMC", upload_date="year")
for i, v in enumerate(y["items"]):
    print(i, v["metrics"]["plays"], v["extra"]["title"])
video_id = y["items"][0]["extra"]["source_id"]
Enter fullscreen mode Exit fullscreen mode

Note that the two search sorts are not the same thing: here Bilibili is ordered by plays, while YouTube's results are in the platform's own order within the upload-date window. Choose videos by hand if you want a fair comparison, and write down how you chose them.

Step 2: video-level numbers

Video detail gives engagement counts. The catalog lists Bilibili's as plays, likes, coins, saves, comments, shares and danmaku (BV ids only), and YouTube's as likes, comments and views, plus channel subscribers.

bv = get("/v1/raw/bilibili/video", bvid=bvid)
yv = get("/v1/raw/youtube/video", video_id=video_id)
print(bv["metrics"])
print(yv["metrics"])
Enter fullscreen mode Exit fullscreen mode

Bilibili's coins and danmaku (on-screen bullet comments) have no YouTube equivalent, so only compare the fields both sides have.

Step 3: pull comments

Both comment endpoints return items, a cursor and has_more:

  • GET /v1/raw/bilibili/comments: bvid, cursor. Top-level comments, hottest first, 20 per page. Pass the previous cursor back unchanged. IP location is not available.
  • GET /v1/raw/youtube/comments: video_id, sort (top or newest), cursor. About 20 per page. 
def pages(path, id_param, id_value, max_pages=5, **extra):
    out, cursor = {}, None
    for _ in range(max_pages):
        params = {id_param: id_value, **extra}
        if cursor:
            params["cursor"] = cursor
        page = get(path, **params)
        for c in page.get("items", []):
            out[c["cid"]] = c
        if not page.get("has_more"):
            break
        cursor = page["cursor"]
    return list(out.values())


bili = pages("/v1/raw/bilibili/comments", "bvid", bvid)
yt = pages("/v1/raw/youtube/comments", "video_id", video_id, sort="top")
Enter fullscreen mode Exit fullscreen mode

Both sides use top/hottest-first ordering here, so we are comparing the most visible comments, not a random sample. That choice shapes every number below.

Step 4: compare

Each comment has text, likes, reply_count and created_at on both platforms, so a shared table is easy:

import pandas as pd

def frame(comments, platform):
    df = pd.DataFrame(
        {"platform": platform, "text": c["text"], "likes": c["likes"], "replies": c["reply_count"]}
        for c in comments
    )
    df["chars"] = df["text"].str.len()
    return df

df = pd.concat([frame(bili, "bilibili"), frame(yt, "youtube")])
print(df.groupby("platform")[["likes", "replies", "chars"]].agg(["count", "median", "max"]))
Enter fullscreen mode Exit fullscreen mode

Character counts are not directly comparable across languages (a Chinese character carries more meaning than an English one), so read chars as a rough length indicator within a platform, not across them. For content-level comparison, the next step is to run the same analysis on both: translate or use a multilingual model for sentiment, extract the most common terms, or just read the top ten by likes on each side:

for name, g in df.groupby("platform"):
    print(f"\n{name}")
    print(g.sort_values("likes", ascending=False).head(10)[["likes", "text"]].to_string(index=False))
Enter fullscreen mode Exit fullscreen mode

Illustrative output shape (not real data):

          likes                 replies               chars
          count median max      count median max      count median max
platform
bilibili     <n>    <x>  <y>      <n>    <x>  <y>      <n>    <x>  <y>
youtube      <n>    <x>  <y>      <n>    <x>  <y>      <n>    <x>  <y>
Enter fullscreen mode Exit fullscreen mode

Caveats worth writing in your own report

  • Sample size and ordering. A few pages of top comments from one video per platform says little about either platform. Use several videos per side.
  • Time. YouTube comment times are UTC, converted from YouTube's relative "3 days ago" and only as precise as the unit named; Bilibili's are Beijing time. Don't compare timing at fine granularity.
  • Language. Any text comparison needs translation or a multilingual model, with its own error.
  • Fields differ. Bilibili has no comment IP location here; YouTube comments carry a creator-heart flag. Compare shared fields only.

Cost

Per the catalog: Bilibili search, video and comments are $0.5 per 1,000 calls each; YouTube search, video and comments are $1 per 1,000 calls each. Calls with no results aren't charged. The spent counter above adds up meta.charged_usd so you can see what each run cost. More on the pricing page.

This article was written with the help of AI.

Top comments (0)