Get Weibo hot searches and posts as JSON with Python (no login)

#python #webscraping #china #showdev

Weibo's hot search list (微博热搜) shows what China is talking about right now. In my runs, the top of the list was completely different on each of the three days I checked, so it is useful for brands, journalists and researchers who follow it over time. On the desktop site, keyword search and user timelines sit behind a login wall, which makes collecting the data harder than it looks.

This post shows how to get the hot search list, posts by keyword, comments and creator profiles from Python, without a Weibo account.

Disclosure: I built the tool used below. It runs on Apify and costs $1 per 1,000 rows.

What you can get

Hot searches: rank, keyword, heat and a label (new, hot, boiling, exploding, pinned)
Posts by keyword, many keywords in one run: text, likes, reposts, comments, topics, author followers, and a flag for paid posts
Comments of any post
Creator profiles, with their latest posts if you want them
The popular feed by category

Step 1: Install the client

pip install -U apify-client

You need a free Apify account for the API token. The free plan gives $5 of usage every month, no credit card needed. Find the token in Apify Console under Settings > API & Integrations.

Step 2: Get the hot search list

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

run = client.actor("serenido/weibo-scraper").call(run_input={
    "mode": "trending",
    "trendingList": "hot_searches",
    "maxResults": 50,
})

for row in client.dataset(run.default_dataset_id).iterate_items():
    heat = row["heat"] if row["heat"] is not None else "-"
    label = row["label"] or ""
    print(f'{row["rank"]:>2}  {heat:>9}  {label:<6} {row["keyword"]}')

The result

The first lines from a real run on October 14 (50 hot searches in 7 seconds):

 0          -  pinned 到2035建成健康中国
 1    1522977  new    腾讯打算拿什么撬动红果近2亿日活
 2     978776  hot    熬夜对人最直观的影响
...

In English: "Build a Healthy China by 2035", "How Tencent plans to take on Hongguo's nearly 200 million daily users", and "The most visible effects of staying up late".

Rank 0 is the pinned topic Weibo places on top. It has no heat number.

Posts by keyword

Switch the mode and pass your keywords:

run_input = {
    "mode": "search",
    "keywords": ["咖啡", "iPhone 17"],
    "includeComments": True,
    "maxCommentsPerPost": 20,
}

Each row has a type (post, comment, creator or hotSearch), so you can split them after the run.

Know the limits

The tool sees Weibo the way a logged-out visitor does. That keeps it simple, and it also sets the limits:

Search: Weibo shows visitors the first page of each search tab (Top, Latest, Hot, With pictures, With videos). All five together give about 90 different posts per keyword. To follow a topic over time, run it on a schedule, for example every hour with the Latest tab.
Comments: visitors see the top comments of a post (usually about 50), with some replies.
Creators: the latest posts option gives the newest posts Weibo shows to visitors (about 10).
Popular feed: about 30 to 40 posts per category at a time.
All dates are in UTC.

Cost

$1 per 1,000 rows. A row is one post, comment, creator or hot search.

Use it from an AI assistant

The tool also works through Apify's MCP server, so Claude, ChatGPT, Cursor or VS Code can run it for you. Add this server URL to your app: https://mcp.apify.com?tools=serenido/weibo-scraper

DEV Community