Advertisers ask it after every campaign: "did my budget buy real engagement?" PR teams ask it during every pile-on: "is this organic?" Here is how to read a comment export like an analyst — the signals that mean something, the ones that don't, and the honest limit nobody selling "bot scores" will tell you about.
First, retire the word "bot"
Three different things hide behind it, and they leak differently:
Tier 1 — automated accounts. Scripts. They leak automation: machine-even posting intervals, generated-name patterns (Firstname Lastname8123 cohorts), impossible hours held for weeks.
Tier 2 — cheap operated fakes. Real humans, fake accounts, paid per comment. This is the tier that actually floods commercial and political campaigns — and it defeats every automation signal, because the typing is human. What it cannot hide is coordination: the campaign brief synchronizes them. They arrive in bursts, they paraphrase the same talking point, they repeat themselves across posts, they reply to each other to lift the thread.
Tier 3 — professionally managed personas. Full histories, varied interests, natural language. Here is the honest limit: in engagement data alone, these are indistinguishable from real people by design. Anyone selling certainty about this tier is selling vibes.
The signals worth trusting (each with its evidence)
Timing bursts — many distinct accounts commenting on one post inside a tight window. Evidence: the timestamp cluster. One burst is an influencer share; recurring bursts with overlapping membership are a pattern.
Duplicate and near-duplicate text — identical or lightly-edited comments across accounts. Copy-paste with cosmetic edits clusters beautifully.
Talking-point convergence — distinct accounts using the same rare phrase inside a window. Softer than duplication, still leaky.
Cross-post repetition — one account, one text, many posts.
Machine-regular cadence — posting intervals too even for a human (needs several comments per account to mean anything).
Burst-only participation — accounts that exist in your data only inside bursts.
Mutual reply-chains — pairs boosting each other's comments to lift thread ranking.
The two disciplines that keep the analysis honest
Insufficient data is an answer. An account with one comment cannot be judged — not "probably fine," not "suspicious." Insufficient signal. Most accounts in most datasets land here, and saying so is what makes the rest of the analysis credible.
Signals describe behaviour, never persons. "This account's comments sit inside two bursts and a duplicate cluster" is evidence. "This account is a bot" is a guess wearing a costume — and a legal exposure if you put it in a client deliverable. Report patterns with evidence attached; let humans decide what to do about them.
The procedure
1. Deduplicate comments first (same comment arrives twice via URL variants).
2. Group by account; count; below threshold -> INSUFFICIENT SIGNAL, stop.
3. Detect bursts per post (window + distinct-account minimum).
4. Cluster normalized text (exact, then near-duplicate, then rare-phrase).
5. Check cadence regularity, cross-post repeats, reply-chain pairs.
6. Levels from accumulated evidence: NO SIGNALS / WEAK / MULTIPLE / STRONG.
7. Print into the deliverable: the coverage block, the evidence per flag,
and what the method CANNOT detect (Tier 3). The limits are the credibility.
Disclosure: I build a tool that runs exactly this — the Engagement Authenticity Audit processes any Facebook comments export ($0.66 per 1,000 unique comments, deduplicated first; free demo with a planted burst you can watch it catch — and a single-comment account you can watch it refuse to judge). Its siblings: the free Page Identity Resolver and Page Compare. The procedure above works without any of them — that's rather the point.
Top comments (0)