DEV Community

Qovia
Qovia

Posted on AI-assisted

Build a daily "what is China talking about" digest with Python (Weibo, Douyin, Zhihu hot lists + comments)

Hot lists are the fastest way to see what the Chinese internet cares about today. Weibo hot search, the Douyin hot list and Zhihu's hot list each move several times an hour, and they often disagree, which is the interesting part. In this post we'll write a small Python script that:

  1. pulls the current Weibo, Douyin and Zhihu hot lists,
  2. finds topics that show up on more than one platform,
  3. finds a Douyin video for one of those topics and grabs its top comments, so you can see the actual reactions,
  4. writes everything to a Markdown file you can send to Slack, email or an LLM.

I'll use Qovia for the data. Disclosure: I'm the maker. The same approach works with any source that gives you these lists as JSON.

Setup

You need Python 3.9+ and requests:

pip install requests
export QOVIA_KEY=sk-qovia-...   # from https://qovia.ai/console/?tab=keys
Enter fullscreen mode Exit fullscreen mode

New accounts get 50 trial calls, which is enough to run this script many times. A call is counted only if it returns data.

Step 1: one helper for every endpoint

Every Qovia endpoint returns the same envelope: data holds the result and meta holds billing info, including what the call cost in meta.charged_usd. So one helper is enough:

import os, requests

BASE = "https://api.qovia.ai"
HEADERS = {"Authorization": f"Bearer {os.environ['QOVIA_KEY']}"}

spent = 0.0

def get(path, **params):
    global spent
    r = requests.get(f"{BASE}{path}", headers=HEADERS, params=params, timeout=60)
    r.raise_for_status()
    body = r.json()
    spent += float(body.get("meta", {}).get("charged_usd", 0) or 0)
    return body["data"]
Enter fullscreen mode Exit fullscreen mode

Tracking charged_usd per call is worth doing from the start. You always know what a run cost, and empty results show up as 0.

Step 2: the three hot lists

The three endpoints take no parameters:

  • GET /v1/raw/weibo/hot_search
  • GET /v1/raw/douyin/hot_search
  • GET /v1/raw/zhihu/hot_list
weibo  = get("/v1/raw/weibo/hot_search")["items"]
douyin = get("/v1/raw/douyin/hot_search")["items"]
zhihu  = get("/v1/raw/zhihu/hot_list")["items"]
Enter fullscreen mode Exit fullscreen mode

A Weibo item looks like this (from the endpoint's documented example):

{ "rank": 1, "word": "AG 退役", "heat": 1383391, "label": "新",
  "category": null, "is_topic": false,
  "url": "https://s.weibo.com/weibo?q=AG%20%E9%80%80%E5%BD%B9" }
Enter fullscreen mode Exit fullscreen mode

Weibo's pinned topics come in a separate pinned list next to items, with rank and heat set to null. Douyin items carry word, rank, hot_value, views, video_count, label, pinned and a url; Douyin's pinned entry is the first item in items, with pinned: true and rank/hot_value null, and label can be null on any item, so filter and sort with that in mind. Each payload also has a prov block with the capture time and whether it was served live or from cache, which is handy when you're comparing lists taken a few minutes apart.

Zhihu's list is made of questions in Qovia's common record shape, so the title and rank live under extra: extra.title, extra.rank, extra.heat (plus the display string extra.heat_label), extra.label and extra.url, with answer and follower counts under metrics.

To print the top 10 of each:

def top(items, n=10):
    ranked = [i for i in items if i.get("rank")]
    return sorted(ranked, key=lambda i: i["rank"])[:n]

def zhihu_top(items, n=10):
    qs = [q["extra"] for q in items if q.get("extra", {}).get("rank")]
    return sorted(qs, key=lambda q: q["rank"])[:n]

for name, items in [("Weibo", weibo), ("Douyin", douyin)]:
    print(f"\n{name}")
    for i in top(items):
        print(f"{i['rank']:>2}. {i['word']}")

print("\nZhihu")
for q in zhihu_top(zhihu):
    print(f"{q['rank']:>2}. {q['title']}")
Enter fullscreen mode Exit fullscreen mode

Step 3: topics trending on more than one platform

Hot-list titles are short Chinese phrases, so exact matching misses a lot. A cheap trick that works well enough for a digest is to compare character bigrams:

def bigrams(s):
    s = "".join(ch for ch in s if not ch.isspace())
    return {s[i:i+2] for i in range(len(s) - 1)}

def similar(a, b, threshold=0.4):
    A, B = bigrams(a), bigrams(b)
    return bool(A and B) and len(A & B) / min(len(A), len(B)) >= threshold

overlap = []
for w in top(weibo, 50):
    for d in top(douyin, 50):
        if similar(w["word"], d["word"]):
            overlap.append((w["word"], w["rank"], d["word"], d["rank"]))
Enter fullscreen mode Exit fullscreen mode

Topics that appear on both Weibo (news- and celebrity-heavy) and Douyin (video-first) are usually the ones that will still matter tomorrow.

Step 4: what people actually say

A hot list tells you what is trending, and comments tell you why. First find a video for the topic with GET /v1/raw/douyin/search (param keyword). Results use the common record shape, and the Douyin video id is extra.source_id:

topic = overlap[0][2] if overlap else top(douyin, 1)[0]["word"]

results = get("/v1/raw/douyin/search", keyword=topic)["items"]
video = max(results, key=lambda v: v["metrics"].get("comments") or 0)
VIDEO_ID = video["extra"]["source_id"]
print(f"\n{topic}: {video['extra']['url']}")
Enter fullscreen mode Exit fullscreen mode

Then fetch its comments with GET /v1/raw/douyin/comments. It needs the video's item_id (the aweme_id, 15–20 digits) and supports count (1–50) and cursor:

page = get("/v1/raw/douyin/comments", item_id=VIDEO_ID, count=20)
comments = sorted(page["items"], key=lambda c: c.get("likes") or 0, reverse=True)
for c in comments[:5]:
    print(f"{c['likes']:>6} ♥  {c['text']}  ({c.get('ip_location') or ''})")
Enter fullscreen mode Exit fullscreen mode

Each comment has cid, text, likes, reply_count, created_at, ip_location, is_hot and an author. To page through, pass cursor=page["cursor"] while page["has_more"] is true. Douyin can reorder between pages, so dedupe by cid.

Xiaohongshu works the same way: GET /v1/raw/xiaohongshu/search, then GET /v1/raw/xiaohongshu/comments with a note_id.

Step 5: write the digest

from datetime import date

lines = [f"# China trends, {date.today()}", ""]
lines += ["## On both Weibo and Douyin", ""]
lines += [f"- {w} (Weibo #{wr}) / {d} (Douyin #{dr})" for w, wr, d, dr in overlap] or ["- none today"]
for name, items in [("Weibo", weibo), ("Douyin", douyin)]:
    lines += ["", f"## {name} top 10", ""]
    lines += [f"{i['rank']}. [{i['word']}]({i['url']})" for i in top(items)]
lines += ["", "## Zhihu top 10", ""]
lines += [f"{q['rank']}. [{q['title']}]({q['url']})" for q in zhihu_top(zhihu)]
lines += ["", f"## What people say about \"{topic}\"", "", f"Video: {video['extra']['url']}", ""]
lines += [f"- {c['text']} ({c['likes']} likes, {c.get('ip_location') or 'unknown'})" for c in comments[:5]]
lines += ["", f"_Cost of this run: ${spent:.4f}_"]

open(f"digest-{date.today()}.md", "w", encoding="utf-8").write("\n".join(lines))
Enter fullscreen mode Exit fullscreen mode

Run it with cron or a GitHub Action once a day. A run is five calls: three hot lists, one search and one page of comments. The cost line at the bottom of the digest tells you what each run spent.

Going further

  • Translate and summarize: feed the digest to an LLM with "translate the topics and explain why each one is trending, using the comments". The model gets real comments to work from instead of guessing.
  • Cross-platform search: POST /v1/search searches about 10 kinds of Chinese public sources at once (Weibo, Douyin, Xiaohongshu, Toutiao, WeChat articles, news sites, forums), with date, platform and sentiment filters. It suits brand-mention tracking better than running per-platform searches.
  • Do it from an agent: the same endpoints are available as a remote MCP server at https://api.qovia.ai/mcp. I wrote about that setup here.

Notes and limits

  • Only public content: hot lists, public posts, comments and profiles. No DMs or private accounts.
  • Billing is per call that returns data, from a prepaid balance with no subscription. Empty results and errors are free, and each call's cost is in meta.charged_usd. Details are on the pricing page.
  • Field names come from each endpoint's docs (https://qovia.ai/docs/?utm_source=devto2). There's also an OpenAPI spec if you want typed clients.

If you build something with this, or want a platform that isn't covered, let me know in the comments.


I wrote this with help from an AI assistant and tested the code myself.

Top comments (0)