Somewhere in your last release announcement there is a number with no unit attached to it, no interval around it, and no date stamped on it. Maybe it was 40% faster. Maybe it was 99.99% uptime, or a cost saving, or a benchmark score. Whoever reads that number and has to act on it — a procurement reviewer, a platform lead, an investor, a lender — must decide how much of their own risk to price into your confidence. An industry analysis of how announcements get read as lightweight disclosure documents in construction and development makes the consequence explicit for people who sign checks, and the mechanism is identical in software: the reader is not hunting for your best number, they are hunting for the range you would defend under questioning. This is an argument about the three things every published number needs and almost never has.
The release format nobody in tech copied
On September 1, 2026, the U.S. Census Bureau reported that construction spending in July ran at a seasonally adjusted annual rate of $2,157.6 billion, 0.5 percent below the revised June estimate of $2,167.7 billion. Then it did something no engineering blog has ever done. It attached ±0.8 percent to that change, and marked it with an asterisk meaning the 90 percent confidence interval includes zero — in plain language, there is not enough evidence to conclude the decline happened at all.
Read that again. A statistical agency published a headline movement and, in the same sentence, told you the movement might not exist. Its tables tag every figure with p for preliminary or r for revised. The methodology note states that the data are annualized and adjusted for seasonality but not for price changes, so you know exactly what basis you are standing on.
That is four pieces of information per number: value, basis, uncertainty, vintage. Now compare it to "we cut p99 latency by 40%." One piece of information, three missing, and the reader has no way to reconstruct the other three without asking you.
The most expensive implicit unit in history
On September 23, 1999, NASA lost the Mars Climate Orbiter. The story usually gets compressed into "someone forgot to convert to metric," which is true and almost useless.
Here is what actually happened. Ground software called SM_FORCES produced impulse data for the angular momentum desaturation events the spacecraft performed during its nine-month cruise. The software interface specification required those values in newton-seconds. The file delivered them in pound-force-seconds. Because one pound-force-second is roughly 4.45 newton-seconds, the navigation team's model absorbed about 22 percent of the real effect of every thruster firing, and the error accumulated quietly for months, far too small on any single event to trip an alarm. The orbiter arrived at roughly 57 kilometres altitude instead of the planned 226 and did not survive.
The board's Phase I report on the mishap is worth reading in full, because the uncomfortable part is this: the onboard software was correct. The numbers were correct. Both units were legitimate measures of impulse. What failed was that a bare number crossed an organisational boundary carrying its unit only by convention, and the receiving system trusted the convention.
Every announcement you publish is that boundary. Inside your team, "40% faster" has a shared basis — everyone knows which workload, which hardware, which percentile. Outside, it is a bare number crossing an interface with nothing but convention holding the meaning in place.
The mirror-image failure: turning an interval into a verdict
The unit problem has a twin, and it is arguably worse because it looks like rigour.
In March 2019, Nature published a call by more than 800 scientists to stop treating statistical significance as a verdict. Valentin Amrhein, Sander Greenland and Blake McShane were not asking for a ban on P values. Their argument was about compression loss: sorting results into "significant" and "non-significant" makes people treat two nearly identical estimates as categorically different, and routinely converts "we did not detect a difference" into "there is no difference." They argued for describing the interval and discussing what the values at both ends would mean, rather than collapsing everything into a binary.
Developers perform exactly this compression every week. A benchmark run produces a distribution across hundreds of iterations. The blog post publishes a single ratio. A load test finds no regression above the noise floor, and the changelog says "no performance impact." Absence of a detected regression is not evidence of no regression, and everyone reading your post who has run a benchmark knows it — which is precisely why they discount the claim.
A number is three fields, not one
The fix is unglamorous and takes about ninety seconds per claim. Treat a published measurement as a composite value:
{
"metric": "request_latency_p50",
"value": 126, "unit": "ms",
"delta": { "pct": -40, "ci95": [-34, -45], "method": "bootstrap, 10k resamples" },
"basis": { "n_runs": 500, "instance": "c6i.4xlarge", "corpus": "2026-08" },
"vintage": { "measured": "2026-08-14", "published": "2026-08-19", "status": "preliminary" }
}
In prose, the same discipline turns this:
"Our new engine is 40% faster."
into this:
"Median request time fell from 210 ms to 126 ms across 500 runs on c6i.4xlarge — a 40% reduction, 95% interval 34% to 45%, measured 14 August 2026 against the 2026-08 corpus."
The second version is longer, less quotable, and dramatically more expensive to dismiss. That trade is the entire point.
Revisions are a feature you have to build
Statistical agencies revise published figures on a schedule and mark the vintage so anyone holding an old number knows it has moved. Software teams silently edit the blog post.
A quietly mutated number is worse than a wrong one, because it destroys everyone's ability to reconcile older references to your work. Keep the URL stable, keep an append-only revision note at the top, and never rewrite history in place. If your benchmark methodology changed, say which vintage the reader is looking at. Nobody has ever lost a deal by disclosing that a preliminary figure was refined; plenty have lost one when a reviewer found two contradictory numbers under the same link.
What to attach to every number you publish
- Unit and basis — the workload, percentile, or accounting definition, next to the figure, not in a footnote
- Sample and selection rule — n, and how those runs were chosen or discarded
- An interval and the method that produced it — bootstrap, standard error, min-max across runs; any of them beats none
- Vintage — measurement date, publication date, and status: preliminary, revised, or final
- A reproduction path — hardware, config, dataset, commit hash
- A scope word — measured, modeled, projected, or contracted, because these are four different commitments
Verification cost is the real currency
Every reader who cannot re-derive your number has to pay for the uncertainty themselves, in extra diligence, in a discount applied to your claim, or in a decision they defer. That cost lands on you whether or not you see the invoice.
The organisations whose numbers move real money — statistical agencies, mission control, peer-reviewed journals — figured this out decades ago, and they all landed in the same place: publish the range, name the unit, date the vintage, and revise in the open. It is not a higher standard than engineering deserves. It is the standard engineering already applies to its own instruments, right up until the moment it writes the announcement.
Top comments (0)