500 Weibo Profile Rows Carried No IP Region. 30 of 36 Search Rows Did

I run a visa agency in Bali. The other half of my week goes into Apify Actors, and one of them reads Weibo — 新浪微博, the Chinese network where media, brands and government accounts publish. Weibo has no open public API for any of this and most of the site asks a visitor to log in, so the Actor reads the pages that m.weibo.cn and weibo.com still hand to a logged-out browser.

One of the fields is in the Actor's own title: IP region. Weibo prints it under a post as 发布于四川 and under a comment as 来自河北 — the province the poster was in. For brand monitoring in China it is one of the few geography signals available at all.

I have three weeks of identical daily runs in my logs, so I went looking at how often that field actually arrives. The answer is not "usually" or "sometimes". It is a clean split by which endpoint the row came from.

Four routes, two answers

Every row below went through the same mapper, in the same Actor, and ipLocation is one line of code reading one key. Here is where it landed, across 30 logged runs between 17 October and 6 November plus two verification runs:

route	post/comment rows	ipLocation present	client present
account timeline (profile tabs)	500	0	500
keyword search	36	30	7
post by link (post detail)	4	0	4
comments	5	5	—

Zero out of 500. Not a sampling artifact and not a block — those rows are complete, with full text, engagement, pictures, video URLs and author. The province is simply not in the JSON the profile-timeline container returns, and not in the JSON the post-detail endpoint returns either.

The client column is the exact mirror image. The label Weibo puts next to the timestamp — 微博视频号, 微博网页版, 微博抽奖平台 — is on all 500 timeline rows and all 4 by-link rows, and on only 7 of 36 search rows. The two routes are complementary. Search knows where you were; the profile knows what app you used. Neither knows both.

I ran the cleanest version of this I could build: one run, one minute, three post links and one keyword together, so proxy, session and code are identical by construction.

search:人工智能   ipLocation=湖北            client=None
search:人工智能   ipLocation=广东            client=人工智能超话
search:人工智能   ipLocation=辽宁            client=None
search:人工智能   ipLocation=吉尔吉斯斯坦     client=分享按钮
search:人工智能   ipLocation=山东            client=None
search:人工智能   ipLocation=None           client=None
by-link          ipLocation=None           client=微博视频号
by-link          ipLocation=None           client=微博视频号
by-link          ipLocation=None           client=微博 weibo.com

So if a tool promises you post-level geography on a Weibo account timeline, ask it which endpoint it reads. There is no amount of proxy or retry that makes region_name appear in a response that does not contain it.

That column is not a province column

Fourth row in that output: 吉尔吉斯斯坦 — Kyrgyzstan. Across my 36 search rows the field took 18 distinct values, and only nine of them are mainland provinces (北京, 四川, 山东, 广东, 江苏, 江西, 湖北, 湖南, 辽宁). The rest: 中国香港, plus 美国, 英国, 法国, 俄罗斯, 丹麦, 加拿大, 安哥拉.

Weibo swaps granularity by location: inside China you get the province, outside it you get the country. If you join this column against a province lookup table, every overseas account silently becomes a null, and overseas accounts are exactly the ones you were probably trying to find.

The cheap way to get Chinese geography is comments

All five comment rows in my test carried a region — 河北, 浙江, 四川, 江西, 广东 — because that one comes from a different key (source on the comment object), and the comment endpoint does return it.

That matters for cost, not just coverage. A post row here is $0.003 and a comment row is $0.00135, so if the question is "which provinces are reacting to this", the comments answer it for less than half the price per row — and they answer it about ordinary users rather than the handful of verified accounts keyword search surfaces.

Now the part I got wrong

Same corpus, different puzzle. Twenty-four of those runs sent a byte-identical input — one account, postsPerAccount: 20. Sixteen runs returned 20 post rows. Eight returned exactly 16.

Sixteen, not fifteen or seventeen. Eight times. And the split correlated perfectly with the clock: every 16-row run fired at 20:xx UTC, which is 04:xx in Beijing.

I had a tidy explanation and evidence for it. 人民日报 goes quiet overnight: in all nine small-hours runs the newest post on the profile was 4.5 to 5.6 hours old when the run started, and across 393 unique posts collected over 22 days, exactly one was published between midnight and 07:00 Beijing. The logged-out profile window is a fixed screen per tab, so a thinner night yields a thinner merge. It fit.

It was wrong. The run log says it plainly:

INFO  Account https://weibo.com/u/2803301701 (uid 2803301701): 20 posts
INFO  Done: {"posts":16,"rows":16,"requests":18,"retries":0,"proxy":false}
INFO  [Status message]: 16 rows: 16 posts. stopped at your Max cost per run

The scrape found 20. Sixteen of them got pushed because the run hit a $0.05 cost ceiling I had set myself. 16 × $0.003 = $0.048; a seventeenth row would be $0.051. Every one of the eight short runs had maxTotalChargeUsd: 0.05; every 20-row run had $1 or more. The hour was a perfect confound because that is simply when my cron fires.

Three weeks of real data, a plausible mechanism, a confirmed correlation, and the cause was a number in my own run options. Check statusMessage before you theorise about the website.

What a cost cap actually costs you

What those four rows were worth is not 20%. A 20-row window on this account spans a mean of 17.7 hours of posting (n=17, range 11.1–22.1h); a 16-row window spans about 8.2 hours. Run once every 24 hours and the gap between yesterday's newest post and today's oldest — the part you never see — looks like this:

rows per run	blind window per day
20	2.2h – 7.0h
16	13.0h – 17.5h

Cutting rows by 20% cut daily coverage from roughly three quarters of the day to roughly one third. On a high-volume account a cost cap is not a discount on your data, it is a hole in your timeline, and the hole is non-linear.

Which leads to the thing I would want told to me before scheduling anything: consecutive daily runs of this account share zero posts. Not few — zero, in every one of my ~24h pairs. 人民日报 publishes faster than a 20-row window refills, so a once-a-day cron is a sample, not a feed. On days I ran two or three times I collected 31, 31 and 33 unique posts; on single-run days, 20. If you need the whole timeline of a busy account, run every six hours and deduplicate on id — do not raise postsPerAccount, because the logged-out profile has no page two at any setting.

Which counters you can trust

Worth knowing before you build a chart, because precision here is per-field, not per-row:

• Engagement is exact at every magnitude. Among my 500 rows, 23 values were ≥10,000 — top likes 56,036, top reposts 210,335 — and not one was a round multiple of 1,000.
• videoPlayCount is not. 252 of its 257 non-zero values are exact multiples of 10,000; the five exceptions are all below 7,000 (626, 3,109, 3,570, 4,670, 6,810). Small play counts are integers, large ones are 万 rounded to two or three significant digits. 8280000 means "828万", give or take 5,000.
• authorFollowers was 158000000 in all 500 rows, every day for three weeks. That is 1.58亿 faithfully expanded. You cannot measure follower growth on a mega-account from a logged-out page — the quantum is 100,000.
• Engagement only ever goes up here: of 80 posts captured in two or more runs, 74 grew, 6 were unchanged, 0 shrank. The fastest moved 79,146 → 210,335 reposts in 6.3 hours. Two runs 47 minutes apart returned the identical 16 ids, and 10 of the 16 had different counters.

One last practical note on search: searchSort changes the time window more than it changes the ranking. latest gave me 10 posts spanning two hours; top gave me 20 posts spanning six weeks, oldest dated 14 September. If you are monitoring, you want latest.

Calling it

curl -s -X POST \
  "https://api.apify.com/v2/acts/lergassy~weibo-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H 'content-type: application/json' \
  -d '{"keywords":["人工智能"],
       "postsPerKeyword":30,
       "searchSort":"latest",
       "commentsPerPost":20}'

Rows are billed per row with no start fee, error rows are free, and every run appends a free type: "notice" row with the exact charge — filter it out with type != "notice". The field list lives on Weibo Scraper.

When this is the wrong tool: there is no history. The logged-out profile is one screen per tab, so "all posts since January" is unavailable at any price, from this Actor or any other without a Weibo login. And if your analysis needs the poster's province on timeline posts specifically, the honest answer is that nobody can sell you that — it is not in the response. Get it from search rows, or from the people in the comments.