The number that embarrasses you is never the obvious lie. It's the ordinary-looking one everybody nods past, because it's already in the deck and the deadline was yesterday.
I've spent weeks checking claims for people. The same seven questions catch most of it. Run them on your own numbers before someone else does.
1. Is it a rate or a score?
"98% accurate" is usually a model score, not a rate of anything real. A score is a claim about a dataset. An accuracy rate is a claim about the world. Those are two different sentences.
2. What's the denominator?
"2,000 users" — out of how many? "40% faster" — than what, measured how? A number without its denominator is a feeling with a percentage sign stapled to it.
3. Who chose the sample?
If the lucky cases got to opt in, the sample picked the answer. Ask who was allowed not to show up. The ones who are missing are usually the story.
4. Is it a max, a mean, or a median?
One great run isn't a typical run. If removing your single best case changes the headline, you don't have a headline. You have a best case.
5. Do the version and the units match the claim?
The benchmark you cite, at the version you actually ran, on the hardware you have. Half the time the number is true and the label on it is wrong. Units are where honest people lie by accident: Rs vs $, ms vs s, per-user vs total, monthly vs annual.
6. Does it survive context?
"X% better than the baseline" is only interesting if the baseline is the thing people actually use. A win over a strawman is a loss with good PR.
7. Who else has to be right for this to be true?
Every number rests on someone else's dataset, API, or report. If you can't name the source and open it, you're not making a claim. You're relaying one, and you've taken on its risk without checking it.
None of this is exotic. It's just work nobody wants to do, because it can only end one way: telling you a thing you'd rather not hear.
I do that work for a living now. Hand me a claim set before it ships — a post, a deck, a product page — I trace each number to its own source, and hand back a written verdict: what holds, what doesn't, what I couldn't reach. Named sources, no vibes. One claim set is $20:
https://buy.stripe.com/3cI8wPc6r6JybadcM55kB3v
If your number is about to walk in front of money, that's the moment to check it. Not after.
Top comments (0)