Disclosure: the author built the tool this post ends with (LinkDigest). Every number below is from live runs on 2026-11-07.
"Most users complain about suction" is the kind of sentence an LLM summary gives you. Most is how many posts? Does that complaint exist in any post, word for word? When you ask a model to summarise a dozen posts, the counts and the quotes are claims, not facts.
We needed this to be right. The job was "for one product category, what do people complain about, what do they search for, and what does nobody answer", for sellers researching China (Xiaohongshu/RedNote and Douyin) and Chinese sellers researching overseas buyers (YouTube and TikTok). So we split it. The model groups. Code counts. Code checks every quote.
1. Demand: what people type into search boxes
Search suggestions are the cheapest honest signal of need. Baidu, Taobao and Bing all answer a plain client that says who it is. For robot vacuum the overseas suggestions include robot vacuum troubleshooting, robot vacuum problems, is it worth buying robot vacuum and robot vacuum vs cordless vacuum. For 扫地机器人 the Chinese ones include "is it a waste of money" (智商税), "how much space to leave for the water hookup" and "Roborock vs Narwal". Group them by keyword rules into buy / worth it / problems / compare / price / scenario.
Two filters you'll need. About 30 of the first 80 English suggestions were Bing place templates like … near me and … in prague, and the Chinese side includes "pictures of" and "in English". Neither is a need.
2. Findings, each tied to a verbatim quote
Read the top posts: caption, the text inside images, the spoken words, and the top comments where the platform serves them. Extract findings (pain points, needs, blockers, products, questions), each with a quote. Then let code look for that quote in the source and drop any finding whose quote isn't there:
def locate(quote, sources):
norm = lambda s: "".join(ch for ch in s.lower() if ch.isalnum())
q = norm(quote)
return bool(q) and any(q in norm(s) for s in sources)
3. The model returns ids, not numbers
Each finding gets an id: n3.objections.2 is the second objection in post 3. The model's only job is to say which ids mean the same thing and to name each group. The count is a set of post numbers:
def count(ids, entries):
return len({entries[i]["note"] for i in ids if i in entries})
What broke on the first live run
The model rewrote the ids. Every n3.questions.2 came back as f3.questions.2, nothing matched, and the whole report fell back to ungrouped findings. Rewording the prompt wouldn't have fixed it. Resolving ids by what identifies them (post number, section and index) did:
ITEM = re.compile(r"(\d+)\s*[.\-_:/]\s*(objections|questions|claims|products|purchase_intent)\s*[.\-_:/]\s*(\d+)\W*$")
The search found a video game. The two hottest posts for 扫地机器人 ("robot vacuum") were about a game called Robot Vacuum Simulator. The model now also lists off-topic post numbers. Code removes them from every count, says which posts it removed, and refuses if the model tries to drop half or more.
Translation ran out of tokens. On the overseas run the model was asked to group 127 findings and translate every quote in one reply. It hit the 16k output limit and the run fell back. Translation is now a second, small call over only the quotes code picked to show.
The model never proposed gaps. So code finds them: a group of searches whose words appear in none of the posts' titles, findings or comments is something "people search for and nobody covers".
Results
扫地机器人, Xiaohongshu + Douyin, 12 posts: 2 off-topic posts removed. Hot-water mop washing and sterilisation appears in 8/10 posts, obstacle avoidance and anti-tangle in 6/10, price and after-sales worries in 4/10. Gaps: "is it a waste of money", "Roborock vs Narwal", "space for the dock".
robot vacuum, YouTube + TikTok, 11 on-topic videos: "can't get under low furniture" in 3/11 ("if it's a small area like a dresser…"), hair clogging in 2/11 ("it's totally clogged up with hair."). Roborock is named in 7/11. Gap: people search troubleshooting and problems, and none of the top reviews covers them.
The sample is the top public search results, and top posts can be sponsored. That's why needs and selling points share one section, and the report says so.
One call
This is now an endpoint, LinkDigest's Pain-Point Radar:
curl -X POST https://linkdigest.dev/api/v1/radar \
-H "Authorization: Bearer $LINKDIGEST_API_KEY" -H "Content-Type: application/json" \
-d '{"topic": "robot vacuum", "market": "global", "size": "quick", "language": "en"}'
# poll GET /api/v1/radar/{id} (10–20 min); ?format=md for Markdown
market: "cn" reads Xiaohongshu and Douyin. "global" reads YouTube and TikTok. A 12-post run is about 41–47 credits, and posts already read cost nothing. Docs: https://linkdigest.dev/docs#radar. Weekly public examples (in Chinese): https://linkdigest.dev/zh/chuhai-tongdian
What it can't read: private posts, comment text on Douyin and YouTube (YouTube asks servers to sign in, and we don't use cookies), and Amazon reviews or Reddit.
Top comments (0)